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In  January  1990,  the  Food  Safety  and  Inspection 
Service  (FSIS)  of  the  U.S.  Department  of  Agriculture 
initiated  efforts  to  determine  how  to  implement  the 
Hazard  Analysis  and  Critical  Control  Point  (HACCP) 
system  in  regulated  meat  and  poultry  plants.  HACCP 
is  a  process  control  system  that  identifies  "critical" 
points  in  the  food  production  process  that  should  be 
controlled  to  prevent  food  safety  hazards  (i.e., 
microbiological,  chemical,  and  physical  hazards)  from 
occurring. 

Agency  activities  included  consultations  and  public 
hearings  to  explain  HACCP  and  the  Agency's  efforts, 
workshops  to  develop  model  HACCP  plans,  in-plant 
testing  to  trial  the  specific  plant  HACCP  plans  at  nine 
volunteer  plants,  and  an  assessment  of  the  overall 
experience  of  the  Agency. 

A  series  of  reports  have  been  written  on  the  various 
phases  and  activities  undertaken  by  the  Agency  and 
participants  during  the  study.  This  is  one  of  the 
reports.  For  further  information  concerning  FSIS 
HACCP  activities  contact: 

Dr.  Dorothy  Stringfellow 
HACCP  Office 
1099  14th  Street,  NW 
Room  6912  Franklin  Court 
Washington,  DC  20005 
Phone:  (202)  501-7319 
FAX:  (202)  501-7639 


EXECUTIVE  SUMMARY 


Background 

In  the  late-  1980's  the  Food  Safety  and  Inspection  Service  (FSIS)  began  encouraging  the 
meat  and  poultry  industry  to  implement  the  Hazard  Analysis  and  Critical  Control  Points 
system  (HACCP).  FSIS  began  focusing  on  HACCP  in  1989.  At  that  time,  Agency 
planners  envisioned  HACCP  as  a  voluntary  program. 

The  Agency  set  forth  four  major  initiatives:  (1)  consultations  and  public  bearings;  (2) 
three-day  workshops  to  develop  generic  HACCP  models;  (3)  development  of  plant- 
specific  HACCP  plans  for  selected  product  classes  that  would  be  used  for  in-plant  pilot 
testing;  and  assessment  of  plant-specific  HACCP  plans  in  a  case  study. 

The  Agency's  primary  concerns  involved:  pre-implementation  stages  for  HACCP  models 
in  volunteer  plants;  conformance  of  generic  models  and  plant-specific  plans  with  HACCP 
principles  developed  by  the  National  Advisory  Committee  on  Microbiological  Criteria  for 
Foods  (NACMCF);  functioning  of  plant-specific  HACCP  models  in  volunteer  plants;  and 
functioning  of  a  HACCP-based  inspection  system  in  volunteer  plants. 

FSIS  conducted  workshops  during  1991  and  early  1992  to  develop  generic  HACCP 
models  for  pilot  testing  in  five  process  areas:  refrigerated  foods,  cooked  sausages,  poultry 
slaughter  (young  chickens),  ground  beef,  and  swine  slaughter.  The  model  plans  identified 
Critical  Control  Points  (CCP's)  and  methods  for  controlling,  monitoring,  and  verifying 
control  at  CCP's.  The  model  plans  for  refrigerated  foods  (RF),  cooked  sausages  (CS),  and 
poultry  slaughter  (PS)  were  pilot  tested.  Concurrent  with  the  workshops,  FSIS  published  a 
Federal  Register  notice  to  solicit  volunteers  to  pilot  test  the  generic  HACCP  models 
developed  at  the  workshops.  Nine  establishments  volunteered  for  pilot  testing  -  three  for 
each  of  the  three  process  areas.  These  establishments  were  to  develop  and  implement 
their  own  plant  and  process-specific  HACCP  plans  based  on  the  generic  HACCP  models. 

Scope 

The  HACCP  case  study  was  designed  to  provide  the  Agency  with  tangible  experience  with 
voluntary  HACCP  in  meat  and  poultry  establishments.  The  evaluation  of  plant-specific 
HACCP  plans  involved  three  phases  of  data  collection:  Baseline  (Phase  I), 
Implementation  (Phase  II),  and  Operational  (Phase  III).  In  Phase  I,  FSIS  collected 
baseline  data  for  at  least  three  months.  Each  plant  continued  to  operate  as  it  had  in  the 
past,  while  developing  its  own  specific  HACCP  plan  based  on  the  workshop  generic 
models.  In  Phase  II,  Agency  inspectors  and  plant  personnel  were  trained  in  HACCP. 
Plant  personnel  adjusted  to  the  new  pattern  of  HACCP-controlled  operations,  found  and 
corrected  process  control  discrepancies,  and  adjusted  the  plan  where  necessary.  Phase  III 
began  when  the  Agency  and  plant  management  agreed  that  the  plant-specific  HACCP  plan 
was  ready  to  be  operational. 
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In  the  initial  stages,  each  generic  model  and  each  plant's  specific  HACCP  plan  was 
assessed  using  checklists  based  on  the  HACCP  principles  of  the  NACMCF. 

Quantitative  analyses  were  performed  for  microbiological  organisms  and  indicators,  and 
for  chemtoal  and  physical  factors  relevant  to  microbiological  status.  FSIS  developed 
sampling  plans,  selected  analytical  test  procedures,  and  designed  survey  instruments. 

Microbiological  sampling  and  testing  provide  information  about  a  food 
manufacturer's  process.  Tests  were  selected  to  recover  a  wide  range  of 
pathogenic  and  non-pathogenic  microorganisms  capable  of  growing  in 
product  and  plant  environments  under  varying  conditions  and  at  different 
points  in  the  process. 

Chemical  factors  were  used  to  supplement  the  microbial  testing  and  to  aid 
in  evaluating  any  observed  microbial  changes.  In  CS  plants,  samples  were 
collected  for  determination  of  nitrites  and  brine.  In  addition,  samples  of 
curing  mixtures  were  collected  for  nitrite  testing.  For  poultry  slaughter, 
sampling  of  total  chlorine  in  water  was  done  daily  with  the  water  being 
sampled  as  it  entered  the  chiller. 

"Physical  factors"  refers  to  product  attributes  or  processing  characteristics 
that  have  an  effect  on  the  final  product's  wholesomeness  or  safety.  FSIS 
constructed  a  series  of  data  collection  tasks  for  each  product  type  to  assess 
plant  control  of  the  physical  factors. 

Qualitative  data  were  gathered  from  plant  records,  plant  personnel,  and  inspection 
personnel  using  various  surveys,  questionnaires,  and  data  collection  instruments.  The 
general  concern  was  management  commitment  to  process  controls,  quality,  regulatory 
compliance,  and  HACCP  training.  Qualitative  factors  specifically  related  to  sanitation, 
hygiene,  plant  personnel  attitudes,  and  commitment  to  producing  a  wholesome  and  safe 
product. 

Industry  and  Agency  training  for  HACCP  was  conducted  at  the  pilot  plants  just  before 
Phase  II  to  familiarize  both  plant  and  inspection  personnel  with  HACCP  principles.  The 
training  was  assessed  to  determine  how  well  it  functioned;  to  gain  insight  into  the  training 
needs  of  inspectors  working  in  a  HACCP-based  inspection  system  and  the  content  needed 
in  HACCP  training;  to  begin  consideration  of  the  target  population  for  HACCP  training; 
and  to  learn  about  the  best  formats  for  delivering  such  training. 

Accomplishments 

The  HACCP  model  checklist  was  used  to  compare  each  plant's  specific  plan  against  the 
1989  NACMCF  principles. 
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All  plans  conformed  highly  to  the  1 989  recommendations  of  the  NACMCF, 
with  the  lowest  score  being  97  percent  in  conformance.  In  general,  the  plans 
presented  adequate  information  and  detail  to  provide  the  basis  for  a  HACCP 
plan. 

Significant  quantitative  data  were  collected  on  the  microbiological  status  of  the  product 
and  plant  environment,  including  supplementary  chemical  and  physical  factors. 

Qualitative  data  provided  an  overview  of  the  plant's  processing  or  slaughter  environment 
with  respect  to  the  management's  commitment  to  controlling  the  manufacturing  process 
and  to  quality  and  regulatory  compliance. 

The  HACCP  literature  review,  published  in  April  1994,  surveyed  the  origins  of  HACCP 
and  its  evolution  and  the  application  of  HACCP  in  various  segments  of  the  food  industry. 

HACCP  Workshops  were  conducted  and  generic  models  were  published  in  reports,  also 
in  April  1994. 

Training  surveys  provided  feedback  to  the  Agency  for  use  in  developing  long-range  plans 
for  HACCP  training. 

The  plant  results  gave  valuable  insights  and  lessons  that  have  and  will  continue  to  help 
FSIS  implement  HACCP  over  the  next  several  years.  The  information  and  data  collected 
from  the  voluntary  HACCP  pilot  study  were  used  in  the  development  of  the  Agency's 
Pathogen  Reduction/Hazard  Analysis  and  Critical  Control  Point  systems  proposed 
regulation,  which  was  published  in  the  Federal  Register  of  February  3,  1995.  Data  on 
plan  development  and  implementation  were  invaluable  in  developing  and  analyzing  the 
cost  impact  of  the  proposed  rule.  The  experience  of  volunteer  plants  in  using  generic 
models  that  they  could  adapt  to  meet  their  specific  needs  was  reflected  in  the  proposed 
rule-making.  The  training  experiences  of  the  pilot  study  have  helped  to  shape  the 
Agency's  plans  in  this  area. 
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PREFACE 


The  case  study  reported  here  was  undertaken  to  provide  the  Agency  with  tangible 
experience  with  HACCP  in  federally-inspected  meat  and  poultry  establishments.  It  was  not 
intended  to  determine  HACCP's  effectiveness,  which  has  been  recognized  by  the  scientific 
community,  industry,  and  government  for  many  years. 

The  Agency's  planning  and  activities  for  the  HACCP  study  began  in  1989  and  ran  into 
1993.  This  complex  project  was  overtaken  by  several  events  during  this  period: 

The  National  Advisory  Committee  on  Microbiological  Criteria  for  Foods 
(NACMCF)  HACCP  principles  set  forth  in  1989  were  the  basis  for  the 
study  design;  the  NACMCF  changed  the  application  of  the  principles  in 
1992,  late  in  the  project's  course. 

In  1993  the  NACMCF  published  The  Role  of  Regulatory  Agencies  and 
Industry  in  HACCP,  again  re-defining  the  basic  context  for  considering 
HACCP. 

In  1993  the  USDA  made  a  significant  change  in  policy  from  envisioning 
HACCP  as  a  voluntary  process  to  proposing  that  it  become  mandatory  for 
all  federally-inspected  meat  and  poultry  establishments. 

The  last  event  is  particularly  important.  Before  this  study,  the  Agency's  experience  in  this 
general  area  had  been  limited  to  Total  Quality  Control  (TQC)  systems,  which  are 
voluntary.  The  new  focus  on  mandatory  HACCP  food  safety  systems  has  introduced  many 
additional  questions  as  to  how  HACCP  should  be  implemented  across  the  entire  meat  and 
poultry  industry. 

After  considering  how  to  report  on  a  study  spanning  a  period  in  which  the  regulatory 
context  changed  dramatically,  the  decision  was  made  to  issue  a  complete  report,  with  all 
the  data  presented,  while  focusing  on  the  elements  of  the  study  that  directly  influenced 
Agency  thinking  on  the  Pathogen  Reduction/Mandatory  HACCP  proposed  regulation 
published  in  the  Federal  Register  of  February  3,  1995.  Data  on  plan  development  and 
implementation  were  invaluable  in  developing  and  analyzing  the  cost  impact  of  the 
proposed  rule.  The  experience  of  volunteer  plants  in  using  generic  models  that  they  could 
adapt  to  meet  their  specific  needs  was  reflected  in  the  proposed  rule-making.  The  training 
experiences  of  the  pilot  study  have  helped  to  shape  the  Agency's  plans  in  this  area. 
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CHAPTER  I 


HACCP  ANAL  YTIC  STUDY  OF  THE 
IMPLEMENTATION  OF  HACCP  SYSTEMS  IN  NINE  . 
SELECTED  MEAT  AND  POULTRY  PLANTS 

BACKGROUND: 

In  the  late  1980s,  FSIS  set  as  a  long-range  goal,  the  implementation  of  HACCP  in 
all  meat  and  poultry  inspection  activities.  FSIS  had  begun  planning  for  HACCP  in  1989. 
In  May  1993  Secretary  of  Agriculture  Mike  Espy  directed  that  FSIS  present  him  with  a 
plan  for  making  the  HACCP  system  of  process  control  mandatory  in  the  nations' s 
federally-inspected  meat  and  poultry  plants. 

During  its  initial  planning  phase,  the  Agency  had  identified  the  following  four 
major  initiatives: 

(1)  consultations  and  public  hearings  to  solicit  views  from  FSIS  employees, 
professional  organizations,  consumer  and  other  public  interest  groups,  and 
industry  (January  through  July  1990); 

(2)  workshops  facilitated  by  FSIS  personnel  to  develop  model  or  generic 
HACCP  plans  and  plant-specific  HACCP  plans  for  selected  product  classes 
(February  1991  through  April  1992); 

(3)  development  of  plant  -  specific  HACCP  plans  for  selected  product  classes 
that  would  be  used  for  in-plant  pilot  testing  (June  1991  through  August 
1993);  and 

(4)  assessments  of  the  HACCP  models  and  the  pilot  tests  using  a  case  study 
approach. 

CONSULTATIONS: 

In  1990,  FSIS  held  consultations  and  public  hearings  to  solicit  views  on  potential 
elements  of  its  HACCP  study  from  interested  parties.  Eight  topics  emerged  from  the 
consultations: 

1.  Selection  of  specific  products  and  processes  as  subjects  for  workshops  to 
develop  model  HACCP  plans; 

2.  Objectives  and  format  for  the  workshops; 
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3.  Role  of  the  FSIS  HACCP  Special  Team  —  seven  Agency  employees  with 
strong  backgrounds  in  all  areas  of  inspection  who  would  facilitate  the 
workshops  and  direct  in-plant  training,  testing,  and  assessment; 

4.  '"      Selection  of  volunteer  plants  to  pilot  test  plant-specific  HACCP  plans 

developed  through  use  of  generic  models; 

5.  Criteria  for  assessing  HACCP  plans; 

6.  Follow-up  with  interested  parties  during  and  after  the  pilot  testing; 

7.  Reporting  and  disseminating  the  core  study  conclusions;  and 

8.  HACCP  training  needs  for  both  FSIS  inspectors  and  industry  personnel. 

FSIS  identified  seven  tasks  that  should  be  assessed  during  the  pilot  (See  Table  1). 
Results  from  some  of  these  tasks  were  to  be  reviewed  by  experts  from  outside  of  FSIS  — 
including  other  government  agencies,  industry,  and  academia  —  while  others  were  not. 

TABLE  1:  HACCP  IMPLEMENTATION  ASSESSMENT  TASKS 


External  Review 

Internal  FSIS  Review 

National  Profiles 

Workshop  Evaluation 

HACCP  Model  Checklist 

Training 

Quantitative  Plant  Data 
(Microbiological,  chemical,  and  physical) 

Qualitative  Plant  Data 

Literature  Review 

This  report  will  discuss  the  detailed  findings  of  key  areas.  More  specifically,  the 
national  profiles  present  an  overview  of  the  industry,  the  particular  HACCP  pilot  program 
areas  (Ready-to-eat  Refrigerated  Foods,  Cooked  Sausages,  and  Poultry  Slaughter  (young 
chickens)),  and  each  of  the  nine  volunteer  plant  participants.  The  HACCP  Model 
checklist  was  used  to  compare  each  generic  model  and  each  pilot  plant's  specific  plan 
against  the  1989  National  Advisory  Committee  on  Microbiological  Criteria  for  Foods 
(NACMCF)  principles  (Reference  1).  Quantitative  data  on  microbiological,  chemical,  and 
physical  factors  were  collected  for  baseline  information  and  comparisons  made  with 
operational  data.  The  quantitative  data  were  collected  for  safety-related  areas  and,  where 
appropriate,  at  critical  control  points,  control  points,  and  finished  product.  Quantitative 
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data  were  collected  to  obtain  an  understanding  of  what  might  transpire  during  HACCP 
implementation  and  identify  possible  effects  of  the  implementation  on  the  industry.  The 
qualitative  data  provided  an  overview  of  the  plant's  managerial  environment  (i.e. 
management's  commitment  and  willingness  to  control  the  manufacturing  process  and  its 
commitment  and  attitude  toward  quality  and  regulatory  compliance  based  on  product 
impact  on  consumer  health  and  safety).  The  results  of  the  Workshops  were  presented  in 
four  reports  which  were  published  also  in  April  1994.  (References  2  through  5)  The 
HACCP  Literature  Review  was  published  in  April  1994.  (Reference  6)  The  training 
survey  results  provided  feedback  to  the  Agency  for  use  in  developing  long-range  plans  for 
HACCP  training. 

WORKSHOPS  FOR  DEVELOPING  MODEL  HACCP  PLANS: 

FSIS  conducted  workshops,  during  1991  and  early  1992,  to  develop  generic 
HACCP  models  for  pilot  testing.  The  workshops  were  held  over  a  three-day  period  with 
approximately  one-and-a-half  days  devoted  to  training  in  HACCP  principles  and  one-and-a 
half  days  devoted  to  generic  plan  development.  The  workshops  developed  generic 
HACCP  models  for  five  product  or  topic  areas:  refrigerated  foods,  cooked  sausages, 
poultry  slaughter  (young  chickens),  ground  beef,  and  swine  slaughter.  Only  the  first  three 
model  plans  were  pilot  tested. 

The  Agency  facilitated  the  workshops,  which  industry  technical  representatives 
attended.  The  model  plans  identified  CCP's  and  methods  for  controlling,  monitoring,  and 
verifying  control  at  CCP's.  Participants  in  each  workshop  established  a  Steering 
Committee  to  refine  the  generic  HACCP  plans  in  cooperation  with  FSIS.  FSIS  also 
selected  "subject  matter  experts"  (SME's),  Agency  in-plant  inspection  employees  who 
provided  technical  expertise  and  guidance  on  specific  products  or  processes  to  the 
workshops  and  steering  committees.  Table  2  lists  the  topic  areas,  locations,  and  dates  of 
the  five  workshops,  and  the  number  of  participants  and  observers  in  each  workshop. 
Workshop  participants  represented  industry,  trade  associations,  and  government.  The 
workshops  were  open  to  the  public  for  observation,  on  a  reserved  space  basis.  Time  was 
allotted  for  observer  comments,  but  observers  were  not  allowed  to  participate  in 
developing  model  plans. 
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TABLE  2:  GENERIC  HACCP  MODEL  PLAN  DEVELOPMENT:  Locations,  Dates, 
and  Number  of  Participants  and  Observers 


Model/Topic  Area, 

Location 

Dates 

Number  of 

1  aiLlL.llJan.Lb 

Number  of 
uDScrvcrs 

Refrigerated  Foods 

Baltimore, 

rvl  anH 

IVJ.aI  y  JoiiU 

February  26-28, 
1991 

40 

42 

Cooked  Sausages 

Fort  Worth,  Texas 

May  21-23,  1991 

35 

25 

Poultry  Slaughter 

Atlanta,  Georgia 

August  27-29,  1991 

43 

54 

Fresh  Ground  Beef 

Phoenix,  Arizona 

December  3-5, 
1991 

31 

12 

Swine  Slaughter 

Minneapolis, 
Minnesota 

March  31  -  April 
2,  1992 

30 

6 

5  Topic  Area  Totals 

179 

139 

Participants  -  representatives  from  industry,  trade  associations,  and  other  non-FSIS  government 
Agencies. 

Observers  -  Open  to  general  public  for  observation,  with  time  allotted  for  comments,  but 
observers  were  not  allowed  to  participate  in  model  development. 


More  detailed  findings  of  the  workshops  are  found  in  the  previously  referenced 
reports.  Detailed  information  on  the  five  generic  models  developed  in  the  workshops 
appears  in  the  following  publications:  1)  Generic  HACCP  Model  for  Refrigerated  Foods; 
2)  Generic  HACCP  Model  for  Cooked  Sausages;  3)  Generic  HACCP  Model  for  Poultry 
Slaughter;  4)  Generic  HACCP  Model  for  Ground  Beef;  and  5)  Generic  HACCP  Model  for 
Swine  Slaughter  (References  7  through  11).  The  Agency  conducted  a  literature  review  of 
the  origins  of  HACCP  and  its  evolution.  The  literature  review  discusses  the  application  of 
HACCP  in  various  segments  of  the  food  industry  (Reference  6). 

DEVELOPMENT  AND  ASSESSMENTS  OF  PLANT-SPECIFIC  HACCP  PLANS  AND 
PILOT  TESTS 

This  report  details  the  methodology  utilized  in  conducting  the  nine  plant-specific 
HACCP  plans'  development  and  the  analysis  of  the  individual  plant's  implementation  and 
operation  of  their  respective  HACCP  plan.  Chapter  II  discusses  the  methodology  while 
Chapter  III  presents  the  analysis  and  findings  of  the  HACCP  pilot  program. 
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CHAPTER  II 


ANALYTIC  STUDY  METHODOLOGY  FOR  THE 
IMPLEMENTATION  OF  HACCP  SYSTEMS  IN  NINE 
SELECTED  MEA  T  AND  POUL  TRY  PLANTS 


ANALYTIC  STUDY  METHODOLOGY: 

Generally,  the  goal  of  research  is  to  be  able  to  extrapolate  from  the  study  results  to 
the  population  that  is  being  studied.  However,  many  factors  affect  the  ability  to  conduct 
true  experimentally-designed  research  (see  Campbell,  D.  T.,  and  Stanley,  J.  C,  1963;  and 
Kidder,  L.  H.,  and  Judd,  C.  M.,  1986  -  References  12  and  13).  Examples  are:  resource 
constraints  in  terms  of  dollars  and  personnel,  participation  of  subjects,  length  of  time 
needed  to  conduct  the  research,  and  length  of  time  needed  to  see  changes  in  the 
experimental  group.  Well-controlled  experimental  research  designs  are  the  ideal  way  to 
conduct  research  because  they  allow  random  assignment  of  subjects  to  the  experiment,  so 
that  the  researcher  can  control  the  influence  of  potentially  compounding  variables.  Such 
designs  use  random  selection  (random  sampling)  of  the  subjects  to  be  studied.  There  is  a 
"treatment  group, "  which  is  subjected  to  whatever  is  being  tested,  and  a  "control  group," 
which  is  not  subjected  the  treatment.  However,  this  is  not  always  possible. 

For  practical  reasons,  FSIS  chose  to  use  an  analytic  study  design  which  involved  a 
pretest-posttest  case  study  approach.  The  analytic  study  design  is  defined  by  Dr.  W. 
Edwards  Deming  as  "one  in  which  action  will  be  taken  on  a  cause-and-effect  system  to 
improve  performance  of  a  product  or  a  process  in  the  future"  (Reference  14).  In  an 
analytic  study,  researchers  aim  for  prediction  that  a  particular  process,  product,  or  material 
is  superior  to  others  in  the  future.  Because  there  is  no  existing  universe  in  an  analytic 
study,  the  study  is  conducted  on  judgment  samples  because  the  judgment  of  experts  in  the 
subject  matter  defines  conditions  to  be  studied  and  measurements  to  be  taken  for  each  set 
of  conditions.  Analytic  studies  are  generally  conducted  on  prototypes  to  learn  how  the 
product  and/or  process  will  function  years  later  in  the  field.  The  analytic  study's 
environment  is  usually  not  static,  but  rather  it  is  dynamic  with  the  primary  source  of 
uncertainty  lying  in  identifying  which  variables  will  have  the  most  significant  impact  on 
future  outcomes  of  the  process. 

Agency  resources  for  program  testing  were  limited,  thereby  limiting  the  number  of 
pilot  plants  that  could  participate  in  the  study.  The  Agency  felt  that  a  multilevel, 
incremental  HACCP  pilot  study  would  help  the  industry  and  Agency  better  understand  the 
concept  and  better  prepare  the  Agency  for  future  nationwide  implementation. 
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DATA  LIMITATIONS: 


The  HACCP  case  study  was  not  designed  to  provide  information  on  the  value  of 
HACCP.  The  scientific  community  has  already  established  HACCP's  value.  The  case 
study  was^'undertaken  to  provide  the  Agency  with  tangible  experiences  in  implementing 
HACCP  in  meat  and  poultry  establishments. 

The  limitations  of  this  case  study  approach  are  partly  due  to  the  fact  that  HACCP 
plans  were  pilot-tested  in  nine  plants  that  volunteered.  Because  plants  vary  greatly,  the 
pilot  test  results  cannot  be  extrapolated  to  all  meat  and  poultry  plants.  Inference  can  only 
be  made  to  the  volunteer  plants. 

Testing  was  conducted  on  the  normal  production  system  of  the  plants  with  no 
introduction  of  anomaly  batches  or  lots  to  ensure  that  all  types  of  batches  or  lots  were 
tested.  It  was  assumed  that  many  different  conditions  of  lots  or  batches  would  be 
encountered.  It  was  not  possible  to  test  for  seasonal  variations  or  other  uncontrollable 
variables. 

Because  only  a  limited  number  of  on-site  reviews  were  conducted,  the  analysis  of 
the  qualitative  data  could  be  biased  by  day-to-day  operational  variability.  It  would  have 
been  preferable  to  review  each  plant  more  than  once  to  collect  both  "pre-HACCP"  and 
"post-HACCP"  data,  but  resource  constraints  prevented  this. 

SOLICITATION  OF  PLANT  PARTICIPATION: 

Concurrent  with  the  workshops,  FSIS  placed  in  the  Federal  Register  a  notice  to 
solicit  volunteers  to  pilot  test  the  generic  model  HACCP  plans  developed  at  the  workshops 
(Reference  15).  Because  of  resource  and  time  constraints,  FSIS  pilot  tested  generic 
HACCP  model  plans  for  only  three  areas  —  Refrigerated  Foods,  Cooked  Sausages,  and 
Poultry  Slaughter  (young  chickens).  Nine  meat  and  poultry  establishments  volunteered  for 
pilot  testing  (three  for  each  product  process  area).  These  establishments  developed  and 
implemented  their  own  plant  and  process-specific  HACCP  plans  based  on  the  generic 
HACCP  models. 

Before  testing  the  plant-specific  HACCP  plans,  FSIS  conducted  brief  "site  reviews" 
of  the  volunteer  plants,  checking  layout,  facilities,  processes,  and  products,  and 
determining  what  adaptations  of  the  generic  HACCP  plans  were  necessary  to  make  them 
compatible  with  specific  plant  situations.  Where  necessary,  FSIS  could  reconvene  the 
workshop  steering  committees  to  discuss  any  refinements  of  the  generic  HACCP  plans, 
based  on  the  assessment  of  the  generic  plans'  applicability  to  the  plant's  operation.  After 
review  of  the  Refrigerated  Foods  HACCP  model,  a  change  was  necessary.  Plant-specific 
HACCP  plans  were  then  developed  by  the  volunteer  plants.  In  each  pilot  plant,  the 
Agency  familiarized  plant  employees  and  FSIS  inspectors  with  HACCP  principles. 


6 


FSIS  HA  CCP  TEAMS  AND  EXPERTS 

Prior  to  conducting  the  HACCP  activities,  FSIS  named  subject  area  experts  and 
formed  various  teams,  to  carry  out  specific  HACCP  responsibilities.  They  were  as 
follows:  *' 

o  Special  Team, 

o  Subject  Matter  Experts, 

o  Regional  Coordinators, 

o  Data  Collectors, 

o  HACCP  Inspectors, 

o  Communications  Team,  and 

o  Evaluation  Team. 

Table  3  lists  the  teams  or  subject  area  experts  and  their  responsibilities/activities.  For 
clarity,  the  narrative  of  this  report  will  refer  to  all  activities  as  Agency  activities/efforts 
and  generally  does  refer  to  any  specific  team  or  subject  area  expert. 

TABLE  3:  RESPONSIBILITIES  OF  FSIS  HACCP  TEAMS  AND  EXPERTS 


Team/Subject 
Area  Expert 

Activities 

SpeciaJ  Team 

Developed  genenc  HACCP  model  and  information  notebook  for  assigned 
process  to  increase  knowledge  level  of  special  team  members 

Developed  presentation  matenal  and  workshop  manual  for  HACCP  workshops 

Trained/prepared  subjea  matter  experts  for  participation  in  workshops  through 
one-on-one  conversations  and  in  a  pre-workshop  seminar 

Presented/facilitated  at  workshops  to  develop  generic  HACCP  models 

Represented  FSIS  in  soliciting  volunteer  plants  for  pilot  plant  phase  of  study 

Coordinated  neering  committee  aaivities  to  refine  workshop  generic  models 

Prepared  instructional/handout  material  for  on-site  evaluation  in  seleaed  pilot 
plants 

Conducted  on-site  evaluation 

Assisted  plants  in  writing  plant-specific  HACCP  plans  from  workshop  genenc 
models 

Prepared  instruaional/handout  material  and  conducted  implementation  training 

Performed  monthly  monitoring  visits/completed  checklist/ answered  questions 

Represented  FSIS  in  the  HACCP  study  through  presentations,  distribution  of 
literature,  etc.,  within  and  outside  the  Agency 

Subjea  Matter  Experts 

Communicated  with  Special  Team  members  to  prepare  pre-workshop  seminars 

Attended  pre-workshop  seminars  and  assisted  Special  Team  members  in 
developing  a  training  generic  model  for  the  specific  workshop  process 
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Team/Subject 
Area  Expert 

Activities 

Aided  Special  Team  members  tn  gaining  knowledge  of  specific  processes 

Assisted  in  preparing  for  the  workshops 

Represented  FS1S  In  the  workshops 

Regional  Coordinators 

Coordinated  workshop  logistics,  location,  facilities,  etc. 

Coordinated  pre-workshop  semjnar/ arranged  for  plant  tours 

Communicated  with  subject  matter  experts 

Assisted  in  locating  volunteer  plants  for  pilot  study 

Coordinated  announcement  for  selection  of  data  collectors  for  pilot  plants 

Prepared  data  collector  monitoring  plan  for  specific  pilot  plants 

Coordinated  on-site  evaluation  logistics 

Attended  on-site  evaluations 

Trained  data  collectors 

Communicated  with  data  collectors  during  all  phases  of  the  study 

Coordinated  implementation  training  logistics 

Conducted  monthly  monitoring  visits/completed  checklist/answered  questions, 
etc. 

Communicated  with  HACCP  Special  Team  members  and  responded  to 
questions  relating  to  the  HACCP  pilot  plants 

Data  Collectors 

Attended  on-site  evaluation  sessions 

■ 

Received  data  collector  training 

Collected  qualitative/quantitative  data  during  all  three  phases  of  the  pilot  study 

Interacted  with  pilot  plant  managemeni/supervisors/employees 

Interacted  with  pilot  plant  FSIS  inspection  personnel 

Answered  questions  on  data  collection  for  FSIS  visitors  to  the  plants 

Attended  pilot  plant  implementation  training 

HACCP  Inspectors 

Attended  on-site  evaluation 

Aided  plants  in  understanding  HACCP  for  the  completion  of  the  plant-specific 
HACCP  plan 

Interacted  with  data  collectors 

Assisted  data  collectors,  Special  Team  members,  Regional  Coordinators  in 
gaining  knowledge  of  pilot  plants 

Interacted  with  FSIS  visitors 

Attended  implementation  training 
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Team/Subject 
Area  Expert 

Activities 

Performed  inspection/ determined  compliance/documented  HACCP  tasks  dunng 
implementation  and  operational  phases  of  the  study 

Interacted  with  plant  management  relating  to  the  HACCP  pilot  study 

Communication  Team 

Served  as  rMb  representative  tor  media  at  workshops  and  as  needed 

Prepared  slides,  pamphlets,  papers,  speeches,  news  articles,  etc.,  concerning  the 
HACCP  pilot  study 

Evaluation  Team 

Construaed  HACCP  assessment  plan 

Performed  plant  visits  to  complete  assessment  instruments 

Prepared/ captured/ analyzed  data  colleaors'  information 

Prepared/ captured/ analyzed  HACCP  inspeaors'  information 

Performed  analysis  and  prepared  final  report 

THREE  PHASE  IN-PLANT  PILOT  TESTING: 
Inspection  Personnel: 

There  were  three  phases  of  data  collection  during  the  pilot  testing  of  plant-specific 
HACCP  plans.  They  were  Baseline  (Phase  I),  Implementation  (Phase  II),  and  Operational 
(Phase  III).  During  the  testing,  "data  collectors"  -FSIS  personnel  independent  of  the 
regularly-assigned  inspectors—collected  appropriate  data  on  product  safety-related  factors, 
to  assess  the  functioning  of  the  plant-specific  HACCP  plans  in  operation.  FSIS  and  plant 
participants  agreed  that  pilot  plant  identity  would  not  be  divulged. 

First,  FSIS  collected  baseline  data  for  three  months  prior  to  implementing  the 
plant-specific  plans.  The  plants  continued  to  operate  as  they  had  in  the  past.  During  this 
period  each  pilot  plant  developed  its  own  specific  HACCP  plan,  based  on  the  workshop 
generic  models.  Data  collectors  became  familiar  with  collection  procedures.  Also,  plant 
employees  become  accustomed  to  the  presence  of  extra  Agency  personnel  involved  in  the 
pilot  testing,  reducing  what  are  known  as  the  "Hawthorne  effect"  and  "halo  effect."  The 
Hawthorne  effect  occurs  when  individuals  under  study  are  aware  of  their  "special  status" 
and  change  their  behavior  to  react  "correctly."  The  halo  effect  "refers  to  the  tendency  for 
overall  positive  or  negative  evaluations  of  the  object  or  person  being  rated  to  influence 
ratings  on  specific  dimensions  (Reference  16). 

The  second  phase  of  the  in-plant  pilot  study  was  implementation  of  plant-specific 
HACCP  plans  in  three  volunteer  plants  for  each  product  type.  This  phase  lasted 
approximately  three  months.  Plant  personnel  adjusted  to  the  new  pattern  of  HACCP 
controlled  operations,  found  and  corrected  process  control  discrepancies,  and  adjusted  the 
system,  where  necessary.  It  was  during  this  phase  that  plant  personnel  and  Agency 
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inspectors  were  trained  in  HACCP.  FSIS  shared  the  baseline  and  implementation  data 
with  plant  management. 

The  third  phase  began  when  the  Agency  and  plant  management  agreed  that  the 
plant-spedfic  HACCP  system  was  ready  to  be  operational.  This  phase  lasted  six  months. 
FSIS  and  the  pilot  plants  agreed  that  data  collected  during  this  phase  would  not  be  shared 
with  the  plant  until  the  final  HACCP  report. 

Tables  4,  5,  and  6  show  the  HACCP.  pilot  plant  schedules  by  topic  area  and  plants. 
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TABLE  4:  HACCP  PILOT  PLANT  SCHEDULE  FOR  REFRIGERATED  FOODS 
(RF) 


PLANT 

DATE 

TASK 

RF-1 

June  11-12,  1991 

On-site  Awareness  Training 

July  22,  1991 

Start  Phase  I-Baseline 

January  10,  1992 

End  Baseline 

January  7  -  9,  1992 

Phase  II  Training 

January  20,  1992 

Start  Phase  II-Implementation 

April  10,  1992 

End  Phase  II 

April  13,  1992 

Start  Phase  Ill-Operational 

October  9,  1992 

End  Phase  m 

RF-2 

July  22  -  25,  1991 

On-site  Awareness  Training 

September  9,  1991 

Start  Phase  I-Baseline 

January  24,  1992 

End  Baseline 

January  14,  1992 

Phase  II  Training 

January  27,  1992 

Start  Phase  II-Implementation 

April  17,  1992 

End  Phase  II 

April  27.  1992 

Start  Phase  Hi-Operational 

October  16,  1992 

End  Phase  m 

RF-3 

August  12  -  14,  1991 

On-site  Awareness  Training 

August  18,  1991 

Stan  Phase  I-Baseline 

April  17,  1992 

End  Baseline 

March  24  -  26,  1992 

Phase  II  Training 

April  20,  1992 

Start  Phase  II-Implementation 

September  28,  1992 

End  Phase  II 

October  5,  1992 

Start  Phase  Ill-Operational 

April  16,  1993 

End  Phase  HI 
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TABLE  5:  HACCP  PILOT  PLANT  SCHEDULE  FOR  COOKED  SAUSAGE  (CS) 


PLANT 

DATE 

TASK 

CS-1*' 

October  15  -17,  1991 

On-site  Awareness  Training 

January  20,  1992 

Start  Phase  I-Baseline 

April  17,  1992 

End  Baseline 

April  28  -  30,  1992 

Phase  II  Training 

May  4,  1992 

Start  Phase  Il-lmplementation 

September  11,  1992 

End  Phase  II 

September  14,  1992 

Start  Phase  Hl-Operationa] 

March  5,  1993 

End  Phase  m 

CS-2 

January  21  -  23,  1992 

On-site  Awareness  Training 

February  24,  1992 

Stan  Phase  I-Baseline 

October  23,  1992 

End  Baseline 

October  19  -23,  1992 

Phase  II  Training 

November  2,  1992 

Start  Phase  II-Implementation 

February  19,  1993 

End  Phase  II 

February  22,  1993 

Stan  Phase  Ill-Operational 

August  20,  1993 

End  Phase  m 

CS-3 

April  20  -  29,  1992 

On-site  Awareness  Training 

June  8,  1992 

Stan  Phase  I-Baseline 

September  30,  1992 

End  Baseline 

October  19  -  23,  1992 

Phase  II  Training 

November  2,  1992 

Stan  Phase  II-Implementation 

February  12,  1993 

End  Phase  II 

February  15,  1993 

Stan  Phase  Ill-Operational 

August  13,  1993 

End  Phase  ID 
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TABLE  6:  HACCP  PILOT  PLANT  SCHEDULE  FOR  POULTRY  SLAUGHTER 
(PS) 


PLANT 

DATE 

TASK 

PS-1 

May  27  -  28,  1992 

On-site  Awareness  Training 

June  1,  1992 

Start  Phase  I-Baseline 

September  18,  1992 

End  Baseline 

August  24  -28,  1992 

Phase  II  Training 

September  21,  1992 

Start  Phase  Li-Implementation 

December  18,  1992 

End  Phase  II 

December  21,  1992 

Start  Phase  Hi-Operational 

June  18,  1993 

End  Phase  m 

PS-2 

May  27  -  28,  1992 

On-site  Awareness  Training 

June  8,  1992 

Start  Phase  I-Baseline 

September  4,  1992 

End  Baseline 

August  24  -  28,  1992 

Phase  II  Training 

September  7,  1992 

Start  Phase  Il-Implementation 

December  4,  1992 

End  Phase  II 

December  7,  1992 

Start  Phase  El-Operational 

June  11,  1993 

End  Phase  EH 

PS-3 

May  27  -  28,  1992 

On-site  Awareness  Training 

June  8,  1992 

Stan  Phase  I-Baseline 

October  16,  1992 

End  Baseline 

October  5  -  9,  1992 

Phase  II  Training 

October  18,  1992 

Start  Phase  Il-Implementation 

February  26,  1993 

End  Phase  II 

February  28,  1993 

Start  Phase  Ill-Operational 

August  27,  1993 

End  Phase  EI 
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Plant-Specific  Plan: 


One  of  the  pilot's  purposes  was  to  gain  experience  in  going  from  generic 
HACCP  plans  to  plant-specific  HACCP  plans  and  processes.  The  information  would 
assist  *he  Agency  in  its  endeavors  to  develop  a  proposed  HACCP  regulation  based  on 
experience  rather  than  theory.  In  the  generic  plans,  identification  of  critical  control 
points,  requirements  for  critical  limits,  monitoring,  corrective  action,  record  keeping, 
and  verification  were  given  in  broad  terms.  Each  pilot  plant  developed  its  own  plant- 
specific  HACCP  plan,  which  reflected  the  individual  product,  processing,  and 
distribution  conditions  of  that  plant.  This  flexibility  allowed  the  generic  plans  to  be 
adapted  to  the  unique  circumstances  of  each  pilot  plant. 

Agency  In-Plant  Activities  during  Phases  I,  II  and  III: 

During  Phase  I,  inspection  at  the  plants  was  conducted  as  normal,  while  the 
plant  developed  its  plant-specific  HACCP  plan  and  FSIS  collected  baseline  data.  Once 
the  baseline  phase  was  completed,  the  plant  operated  its  HACCP  system  for  three 
months  during  the  implementation  phase  and  six  months  during  the  operational  phase. 
The  Agency  utilized  the  regularly-assigned  inspectors  throughout  the  test  period,  who 
verified  that  routine  compliance  standards  were  met.  Verification  was  defined  as: 
methods,  procedures,  and  tests  used  to  determine  if  the  HACCP  system  is  in 
compliance  with  the  HACCP  plan.  Agency  verification  tasks  were  developed  from  the 
generic  plans. 

FSIS  inspectors  also  performed  evaluation  tasks,  which  included  checking 
records  and  looking  for  trends.  Inspectors  evaluated  records  for  deviations  or 
deficiencies  on  scheduled  tasks  to  determine  compliance  with  critical  limits  identified 
in  the  HACCP  plan,  and  with  regulatory  standards. 

The  remainder  of  the  chapter  discusses  the  methodological  designs  used  to 
determine  what  would  be  measured,  how  it  would  be  measured,  questionnaire  designs, 
sampling  plans  and  timeframes,  and  data  sources.  The  order  of  the  discussions  is: 

1 .       National  Profile  Methodology 


2 


HACCP  Model  Checklist, 


3 


Quantitative  Plant  Data  for 


A. 


Microbiological  Factors, 


B. 


Chemical  Factors,  and 
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c. 


Physical  Factors 


4. 


Qualitative  Data  From  Plant  Management  Assessment  for 


A. 


Commitment  to  control  manufacturing  process,  and 


B. 


Commitment  to  quality  and  regulatory  compliance 


5. 


Training 


A. 


FSIS 


B. 


Plant 


NATIONAL  PROFILE  METHODOLOGY: 

The  National  Profiles  provide  a  picture  of  the  industry,  the  three  segments 
(Refrigerated  Foods,  Cooked  Sausage,  and  Poultry  Slaughter),  and  the  nine  volunteer 
plants  by  various  production  and  inspection  characteristics.  Various  Agency  databases 
were  used  to  develop  the  National  Profiles.  These  included  the  Performance  Based 
Inspection  System  (PBIS),  the  Special  Survey  of  Refrigerated  Foods,  and  the  Animal 
Disposition  Reporting  System  (ADRS). 

The  profiles  consisted  of  such  characteristics  as  annual  production/slaughter 
volume,  plant  square  footage  (size),  and  level  of  compliance  to  PBIS  inspection  tasks. 
PBIS  and  ADRS  data  were  used  to  relate  health  and  safety  activities  for  each  topic 
area  and  to  report  these  data  by  volume  and  size  characteristics. 

PBIS  contains  the  following  four  major  components:  Plant  profiles 
(characteristics  such  as  size,  volume,  and  management  officials);  the  Inspection  System 
Guide,  a  compilation  of  inspection  tasks  to  measure  industry  compliance  to  inspection 
standards  for  all  processing  and  allied  slaughter  activities;  a  Corrective  Action  System 
(procedures  for  correcting  deficiencies);  and  an  automated  data  support  system  that 
schedules  inspection  activities  and  records  results  of  FSIS  inspections. 

FSIS  conducted  the  Special  Survey  of  Refrigerated  Foods  from  September 
through  November  1990  to  obtain  information  on  the  number  of  FSIS-inspected 
refrigerated  foods  plants  and  the  types  of  products  they  produced. 

The  ADRS  uses  various  forms,  one  of  which  is  FSIS  Form  MP-513  (Poultry 
Inspection— Daily  Summary),  that  include  the  condemnation  rates  of  slaughtered  young 
chickens. 
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HACCP  MODEL  CHECKLIST  METHODOLOGY: 


Each  generic  model  and  each  pilot  plant's  specific  HACCP  plan  were  assessed 
using  the  1989  HACCP  principles  of  NACMCF. 

The  assessment  of  the  generic  HACCP  plans  and  plant-specific  plans  focused 
on  the  plans  themselves  and  not  on  implementation  of  the  plans  by  the  plants.  Two 
checklists  of  50  questions  (Yes/No)  were  developed.  The  checklist  for  plant-specific 
plans  was  expected  to  have  answers  to  all  50  questions.  Generic  models,  by  their  very 
nature,  however,  did  not  contain  such  specificity,  and  their  checklist  contained 
additional  scoring  category  of  "not  applicable."  Each  generic  model  and  plant-specific 
plan  was  scored  against  the  50  questions.  Each  yes  answer  received  a  positive  score  of 
two  points  (+2).  Questions  not  addressed  received  a  negative  score  of  two  (-2). 
Certain  questions  required  information  specific  to  a  plant  and  its  product,  which  were 
not  likely  to  be  available  when  the  generic  model  plan  was  developed.  In  these  cases, 
the  questions  were  not  fully  applicable  and  no  score  was  given.  When  a  question  was 
partially  addressed,  a  positive  score  of  one  (+1)  was  awarded.  The  minimum 
conformance  level  to  the  principles  outlined  by  the  NACMCF  was  +50  points. 

QUANTITATIVE  PLANT  DATA  METHODOLOGIES: 

Quantitative  methodologies  were  developed  to  assess  microbiological,  chemical, 
and  physical  factors.  For  each  of  the  three  areas,  FSIS  identified  specific  factors, 
developed  sampling  plans,  assigned  analytical  test  procedures,  and  designed  survey 
instruments.  The  Agency  also  gave  specific  instructions  for  collecting,  mailing,  and 
receiving  samples,  and  for  following  laboratory  protocols.  A  survey  questionnaire  was 
designed  for  the  physical  factors.  The  factors  were  identified  without  reference  to  the 
HACCP  models  because  the  models  were  not  developed  at  the  time  the  study  design 
and  plan  were  completed.  The  factors  were  selected  based  on  their  relation  to  health 
and  safety  and  once  the  HACCP  models  were  developed,  further  assessment  was  made 
as  to  the  appropriateness  of  testing  for  each  factor  at  a  specific  critical  point. 

Selection  of  Microbial  Organisms: 

Microbial  sampling  and  testing  were  conducted  to  provide  information  about  a 
food  manufacturer's  process.  For  this  purpose,  a  wide  range  of  microbial  tests  was 
selected  to  recover  pathogenic  and  non-pathogenic  microorganisms  capable  of  growing 
in  both  the  product  and  the  environment  under  varying  conditions. 

The  eight  microbiology  factors  were: 

1.  Aerobic  Plate  Count  @3  5 °C, 

2.  Aerobic  Plate  Count  @20°C, 

3.  Coliforms, 
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4. 
5. 
6. 
7. 
8. 


Escherichia  coli  (Biotype  I), 
Staphylococcus  aureus, 
Gas  Forming  Anaerobes, 
Salmonella  species. 
Listeria  species, 


All  the  factors  except  Salmonella  were  selected  to  assess  the  processes  in 
refrigerated  and  cooked  sausage  plants.  Listeria  was  selected  to  assess  the  processing 
environment.  In  poultry  slaughter  plants,  Salmonella  was  added  as  a  microbiology 
factor,  and  Listeria  and  APC  @  20°C  were  not  included. 

Brief  Description  of  Microbiology  Factors: 

A  discussion  of  the  choice  of  organisms  to  be  used  in  microbiological  criteria  is 
found  in  the  National  Academy  of  Sciences  (NAS)  study,  "An  Evaluation  of  the  Role 
of  Microbiological  Criteria  for  Foods  and  Food  Ingredients"  (Reference  1 7).  The 
rationale  used  in  that  book  was  incorporated  into  the  FSIS  HACCP  study. 

Aerobic  Plate  Count  @35°C:  A  count  of  total  aerobic  microorganisms 
present  in  a  sample  and  indicative  of  the  bioburden  present.  The  count  will  be 
lowest  just  after  effective  cooking,  but  will  increase  with  time  as  the  surviving 
microorganisms  grow  and  multiply.  The  rate  of  multiplication  increases  when 
the  storage  temperature  rises. 

Aerobic  Plate  Count  @20°C:  Similar  to  the  definition  above,  but  the  lower 
temperature  of  incubation  favors  growth  of  psychrotrophic  (cold  tolerant) 
spoilage  organisms  over  those  which  might  cause  human  illness.  Very  high 
APC  @20°C  counts  (>  10,000,000  CFU/g)  indicate  that  product  is  approaching 
the  end  of  its  useful  shelf-life.  This  will  ultimately  occur  with  all  perishable 
food  products  stored  under  refrigeration. 

Coliforms:  Coliform  bacteria,  normally  present  in  meat,  poultry,  and  many 
other  raw  food  products,  are  readily  destroyed  during  heat  processing. 
Excessively  high  coliform  counts  may  result  from  massive  contamination, 
process  failure,  or  from  extended  processing  delays,  or  from  improper  storage 
temperature  (time-temperature  abuse). 

Escherichia  coli:  This  coliform  species  are  associated  with  warm  blooded 
animals,  including  humans,  includes  both  pathogenic  and  non-pathogenic 
biotypes  and  is  relatively  heat  sensitive.  The  presence  of  E.  coli  in  a  heat- 
processed  food  indicates  underprocessing  and/or  postprocessing  contamination 
through  equipment  or  utensils,  by  persons  handling  the  cooked  food,  or  from 
cross-contamination  with  raw  foods.  Occasionally,  a  few  E.  coli  organisms 
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reach  the  final  product,  even  under  reasonably  good  manufacturing  practices. 

Staphylococcus  aureus:  The  presence  of  S.  aureus  in  food  indicates  direct  or 
indirect  contamination  from  human  or  animal  sources.  Foodborne  illness  is  the 
"  ' result  of  the  ingestion  of  the  enterotoxin  produced  when  S.  aureus  reach  high 
levels  (>  105  cells/g).  The  enterotoxin  is  not  destroyed  by  heating. 

Gas  Forming  Anaerobes:  Gas-Forming  Anaerobes  (GFAs)  include  spore- 
forming  Clostridia,  such  as  Clostridium  perfringens.  Adequate  heat  treatment 
destroys  the  vegetative  Clostridia  in  meat,  but  spores  may  survive.  If  meats 
containing  these  spores  are  held  at  temperatures  between  68  and  122°F,  the 
spores  may  grow  and  multiply,  creating  a  potential  health  hazard.  High  levels 
of  GFAs  in  a  cooked  product  generally  indicate  improper  cooling  or  prolonged 
temperature  abuse. 

Listeria  species:  Some  species  of  the  genus  Listeria,  which  includes  the 
pathogenic  species  Listeria  monocytogenes  and  several  non-pathogenic  species, 
can  grow  and  multiply  at  refrigeration  temperatures,  making  this  an  excellent 
indicator  of  the  effectiveness  of  cleaning  and  sanitization  procedures  in 
processing  environments. 

Salmonella  species:  High  levels  of  Salmonella  in  raw  products  immediately 
after  slaughter  may  be  indicative  of  improper  cooling,  temperature  abuse,  or 
ineffective  scalding,  evisceration,  or  washing.  Microbiological  testing  for 
Salmonella  was  performed  only  in  the  poultry  slaughter  plants. 

Microbiology  Analytical  Methods: 

The  analytical  methods  used  for  the  various  microbiology  factors  are  described 
in  F SIS' s  Microbiology  Laboratory  Guidebook  Section  3  (1974),  (Reference  18)  with 
the  following  modifications: 

1 .       Substitution  of  3M™  petri  film  for  enumeration  of  coliforms, 
E.coli  and  APC. 
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2.       Salmonella  analytical  method  as  described  in  Food  Technology, 
1969,  23:80-85  (Reference  19)  and  section  4,  MLG  as  modified  by  the  substitution  of 
DMLIA  plating  media  Journal  of  Food  Protection,  1988,  51:391-396  (Reference  20). 

Chemical  Factors  Methodologies: 

The  chemical  factors  in  the  pilot  testing  were  not  used  as  indicators  of  chemical 
hazards,  but  rather,  to  supplement  the  microbial  testing  and  evaluate  any  observed 
microbial  changes. 

Chemistry  sampling  and  testing  were  conducted  for  the  cooked  sausage  plants 
and  poultry  slaughter  plants.  The  chemistry  testing  methodology  called  for  performing 
the  following  tests  on  applicable  product  by  topic  area. 

Chemistry  Factors  for: 

Refrigerated  Foods 

Chemistry  sampling  was  not  done  at  the  three  refrigerated  foods  plants  because  their 
products  were  not  amenable  to  the  selected  chemistry  factors. 

Cooked  Sausage 

Nitrites:  Cooked  sausages  are  allowed  by  regulation  to  contain  a  restricted  amount  of 
sodium  or  potassium  nitrite.  The  use  of  nitrites  at  a  level  not  to  exceed  156  ppm 
(parts  per  million)  provides  a  barrier  in  the  formulation  of  the  product  in  addition  to 
refrigeration,  against  the  growth  of  certain  microorganisms.  Excessive  amounts  of 
these  curing  materials  are  toxic. 

Salt  (Brine):  Moisture  and  salt  concentrations  were  determined  so  that  the  brine 
content  could  be  calculated.  The  brine  content  is  the  concentration  of  salt  in  the 
aqueous  fraction  of  the  product.  It  is  calculated  by  dividing  the  amount  of  salt  by  the 
amount  of  salt  plus  moisture.  The  brine  content  of  a  product  relates  to  the  water 
activity  of  the  product,  which,  in  turn,  influences  the  extent  of  microbial  growth 
possible  in  the  product. 

Poultry  Slaughter 

Chlorine  in  Processing  Water:  Chlorinated  water  is  used  to  reduce  the  level  of 
microorganisms  in  carcass  chillers  and  in  reprocessing  procedures.  Chlorine  test  kits 
were  used  to  verify  the  concentration  of  chlorine  in  the  wash  or  spray  water. 
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Chemistry  Analytical  Methods: 

Type  II  protein  content  was  determined  by  the  Kjedahl  method,  moisture  by  the 
Forced  Draft  Oven  Method,  and  salt  by  the  Volhard  Procedure.  Brine  was  determined, 
by  calculation,  from  salt  and  moisture  values.  Nitrite  was  determined  by  a 
modification  of  the  Griss  Method.  The  percentage  of  nitrite  in  each  new  batch  of 
curing  material  was  also  checked  by  laboratory  analysis.  For  poultry  slaughter 
facilities,  tests  of  the  concentration  of  total  chlorine  in  processing  water  were 
performed  in-plant,  by  an  inspector  using  a  HACH  model  CN-21P  test  kit,  which  FSIS 
had  evaluated  and  found  to  be  sufficiently  accurate  within  the  expected  working  range. 

SAMPLING  PLANS: 

The  sampling  plan  called  for  collecting  sample  sets  for  both  microbiology  and 
chemistry  on  30  to  50  different  days  during  the  baseline,  implementation,  and 
operational  phases.  Samples  were  analyzed  at  the  FSIS's  Mid-Western  Laboratory. 

The  microbiological  results  from  Phase  I  were  shared  with  these  plants  at  the 
beginning  of  Phase  II  and  those  from  Phase  II  were  shared  as  they  became  available. 
Phase  III  began  when  both  the  Agency  and  plant  management  agreed  that  the  HACCP 
system  had  been  fully  implemented.  Due  to  a  five-to  six-week  lag  time  between 
sample  collection  and  feedback,  the  microbiology  and  chemical  results  were  of  little 
use  to  plant  management  for  process  control  decisions.  However,  the  shared  results 
were  useful  for  verification  tasks  performed  by  FSIS  personnel.  The  Phase  III  results 
were  shared  with  the  plants  when  the  case  study  reports  were  completed. 

Microbiological  Sampling  Plan: 

The  samples  were  taken  at  selected  locations  during  processing  for  the  baseline 
(Phase  I),  implementation  (Phase  II),  and  operational  (Phase  III)  phases  of  the  study. 
Samples  were  to  be  frozen  and  then  shipped  to  the  FSIS's  Mid- Western  Laboratory. 

For  refrigerated  foods  plants,  samples  were  collected  for  the  first  six  microbial 
factors  at  three  points  in  the  production  process:  1)  before  cooking;  2)  after  cooking 
and  chilling;  and  3)  the  final  packaged  products.  Cooked  sausage  plant  samples  were 
collected  for  the  first  six  microbial  factors  at  three  points:  1)  after  stuffing,  but  before 
cooking;  2)  after  cooking  and  chilling;  and  3)  the  final  packaged  product.  All 
sampling  locations  were  chosen  to  provide  information  before  and  after  processing 
steps  which  are  known  to  influence  the  microbiological  condition  of  the  final  product. 

Listeria  species  testing  was  performed  on  four  different  equipment  surfaces  on 
an  alternating  basis.  Swabs  for  Listeria  were  taken  and  reported  as  positive  or 
negative,  but  were  not  quantified. 
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Salmonella  testing  was  performed  only  in  the  poultry  slaughter  area  at  four 
different  locations:  just  prior  to  entry  into  the  scald  tank;  from  the  transfer  table  after 
hock  cutting;  after  the  final  washer  and  just  prior  to  the  chill  tank;  and  after  the  final 
chill  tank. 

Table  7  displays  the  original  microbiological  sampling  plan,  by  product  type 
and  phase.  The  plan  was  to  collect  sample  sets  on  30  to  50  different  days  during  both 
the  baseline  and  operational  phases.  The  actual  number  of  samples  collected  is  shown 
in  Table  8. 
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TABLE  7:  MICROBIOLOGICAL  SAMPLING  PLAN  BY  PRODUCT  TYPE 
AND  PHASE 


PRODUCT  TYPE 

TOTAL  NUMBER  OF  SAMPLES  TARGETED* 

PHASE  I 
BASELINE 

PHASE  III 
OPERATIONAL 

Refrigerated  Foods  Plants 

550 

550 

Cooked  Sausage  Plants 

600 

600 

Poultry  Slaughter  Plants 

600 

600 

Total 

1750 

1750 

*  Sampling  Targeted  for  30  ro  50  days  with  minimum  samples  per  point  set  at  30  per  phase. 
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TABLE  8:  NUMBER  OF  MICROBIOLOGICAL  SAMPLES  COLLECTED  BY 
PRODUCT  TYPE,  PLANT,  AND  PHASE 


PRODUCT  TYPE/PLANT 

ACTUAL  NUMBER  OF  SAMPLES 
COLLECTED 

PHASE  I 
BASELINE 

pfla.se  in 

OPERATIONAL 

Refrigerated  Foods 

Plant  1 

372 

192 

Plant  2 

209 

123 

Plant  3 

239 

106 

Total 

820 

421 

Cook  Sausages 

Til  1 

Plant  1 

200 

142 

Til       -  i 

Plant  2 

203 

183 

Plant  3 

291 

129 

Total 

694 

454 

Poultry  Slaughter 

Plant  1 

228 

208 

Plant  2 

212 

204 

Plant  3 

264 

200 

Total 

704 

612 
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Chemistry  Sampling  Plan: 

In  cooked  sausage  plants,  samples  were  collected  for  nitrites  and  salt  testing.  Final 
product  samples  were  collected  for  nitrites  and  salt.  In  addition,  samples  of  curing 
mixtures  Were  collected  for  nitrite  testing. 

For  poultry  slaughter,  sampling  of  total  chlorine  in  water  was  done  daily  with  the 
water  being  sampled  as  it  entered  the  chiller. 

Inspectors  were  instructed  to  alternate  morning  and  afternoon  collection.  However, 
collection  was  more  typically  carried  out  at  various  times  during  a  single  shift,  since 
testing  was  generally  done  by  the  same  inspector. 

Table  9  displays  the  number  of  chemistry  samples  collected  for  the  product  type  by  plant. 

TABLE  9:  NUMBER  OF  CHEMISTRY  SAMPLES  COLLECTED  BY  PRODUCT 
TYPE,  AREA,  PLANT,  AND  PHASE 


NUMBER  OF  SAMPLES  COLLECTED 

PRODUCT  TYPE/PLANT 

PHASE  I 

PHASE  III 

Refrigerated  Foods  -  NA* 

Cooked  Sausage 

Plant  1 

58 

38 

Plant  2 

75 

14 

Plant  3 

74 

48 

Total 

207 

100 

Poultry  Slaughter 

Plant  1 

68 

117 

Plant  2 

65 

121 

Plant  3 

43 

71 

Total 

176 

309 

*  The  three  plants'  products  were  not  amenable  to  proposed  refrigerated  food  chemistry  testing. 
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PHYSICAL  FACTORS  METHODOLOGY: 


The  term  "physical  factor"  in  this  study  refers  to  product  attributes  or  processing 
characteristics  that  have  an  effect  on  the  final  product  wholesomeness  or  safety.  The  term 
should  not  be"  "considered  synonymous  with  "physical  hazards"  as  defined  by  HACCP.  The 
physical  factors  identified  for  study  purposes  were  developed  prior  to  the  availability  of  the 
generic  HACCP  models  and  were  intended  to  provide  a  broad  overview  of  the  plant's  controls 
and  operations. 

Physical  factors  are  process  control  oriented.  For  example,  the  temperature  of  a 
product  at  receiving,  or  the  presence  of  foreign  material  in  a  product  during  processing,  are 
product  attributes  that  affect  the  wholesomeness  of  a  final  product.  The  temperature  of  a 
storage  area  or  the  accuracy  of  a  thermometer  are  processing  characteristics  that  affect  the 
wholesomeness  of  a  final  product. 

Observations  made  of  a  plant's  efforts  to  control  physical  factors,  such  as  temperature 
and  presence  of  foreign  material,  give  an  indication  of  a  plant's  control  of  its  manufacturing 
processes.  Plant  control  of  each  physical  factor  was  assessed  through  direct  observation  by  a 
FSIS  data  collector  prior  to  HACCP  implementation  (Baseline  Phase  I)  and  after  HACCP 
implementation  (Operational  Phase  III)  to  determine  if  plant  controls  changed. 

FSIS  constructed  a  series  of  data  collection  tasks  for  each  product  type  (Refrigerated 
Foods,  Cooked  Sausage,  and  Poultry  Slaughter)  to  assess  plant  control  of  physical  factors. 
Each  series  of  data  collection  tasks  contained  several  sets  of  questions.  Plant  control  of 
physical  factors  was  assessed  by  observing  plant  monitoring  activities,  plant  records,  and  plant 
reactions  to  process  deviations. 

Verification  tasks  at  each  physical  factor  were  conducted  by  the  data  collector  to 
ascertain  the  effectiveness  of  plant  control.  For  example,  temperature  and  organoleptic 
wholesomeness  of  product  at  receiving  was  identified  as  a  physical  factor  that  assures 
acceptable  product  is  brought  into  the  plant.  The  data  collector  would  observe  the  plant 
actions  at  receiving  to  determine  if  an  established  procedure  is  followed  when  incoming 
product  is  brought  into  the  plant.  Secondly,  the  data  collector  observed  if  the  plant 
documented  the  findings  when  an  assessment  of  incoming  product  was  made.  Thirdly,  the 
data  collector  determined  if  proper  corrective  action  were  taken  when  unwholesome  product 
was  received  or  was  received  at  too  high  a  temperature.  Lastly,  the  data  collector  verified 
that  product  received  was  in  fact  wholesome  and  at  proper  temperatures  by  conducting  their 
own  temperature  checks  and  wholesomeness  checks. 

Physical  Factors: 

The  physical  factors  for  refrigerated  foods,  cooked  sausage,  and  poultry  slaughter  are 
displayed  in  Table  10.  A  brief  description  of  the  purpose  of  each  factor  by  product  type 
follows.  Both  refrigerated  foods  and  cooked  sausage  plants  had  the  same  five  factors,  while 
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poultry  slaughter  had  three  factors.  The  factors  for  each  product  type  are  as  follows: 
Refrigerated  Foods  and  Cooked  Sausages: 

The  physical  indicators  for  Refrigerated  Foods  and  Cooked  Sausages  are: 

Temperature  -  Maintenance  of  proper  temperature  minimizes  undesirable  microbial 
growth. 

Organoleptic  Wholesomeness/Foreign  Material  -  Organoleptic  signs  of  spoilage  (such 
as:  odor,  slipperiness)  or  contamination  are  indications  of  unacceptable  product,  while 
foreign  materials  (metal,  glass,  etc.)  are  potential  hazards. 

Heat  Treatment/Internal  Temperature/Cooling  Treatment  -  Application  of  appropriate 
heat  increases  shelf-life  and  decreases  the  number  of  microorganisms;  inadequate  heat 
treatment  allows  microorganisms  to  survive.  After  cooking  or  heating,  product  must 
be  cooled  to  a  specified  temperature  in  an  appropriate  amount  of  time. 

Cross-Contamination  -  Prevention  of  cross-contamination  of  cooked  products  by 
contact  with  raw  products. 

Packaging  -  Package  integrity  deters  microbiological  proliferation  and  finished  product 
date  coding  allows  for  the  removal  of  unsafe  product. 

Poultry  Slaughter: 

The  three  poultry  slaughter  physical  factors  were: 

Sanitation  -  Preventing  contamination  in  slaughter  facilities  and  on  equipment  at  pre- 
operational sanitation  and  mid-shift  cleanup;  determining  the  level  of  sanitation  of 
automatic  eviscerating  machinery  and  potential  of  cross-contamination;  and 
determining  the  accuracy  of  sanitary  controls  in  removing  slaughter/processing  debris 
from  equipment,  utensils,  and  structures. 

Contamination  -  Preventing  fecal  contamination  and  cross-contamination  of  product 
and  equipment  from  carcasses  leaking  feces;  effective  manufacturing  process  for 
controlling  visible  contamination  on  carcasses  going  into  the  chiller;  and  effective  plant 
salvage  and  reprocessing  procedures. 

Product  Handling  -  Preventing  condemned  materials  from  being  used  as  human  foods; 
assessing  the  accuracy  of  inspection  helpers  who  respond  to  inspection  instructions; 
assessing  sanitary  practices  of  salvage  and  reprocessed  products;  and  determining  the 
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efficiency  of  product  flow  and  time  and  temperature  conformance. 


TABLE  10:  PHYSICAL  FACTORS  FOR  REFRIGERATED  FOODS,  COOKED 
SAUSAGE  AND  POULTRY  SLAUGHTER  VOLUNTEER  PLANTS 


REFRIGERATED  FOODS 

COOKED  SAUSAGE 

POULTRY  SLAUGHTER 

1.  Temperature 

1.  Temperature 

1.  Sanitation 

2.    Foreign  Material/ 
Organoleptic 
Wholesomeness 

2.    Foreign  Material/ 
Organoleptic 
Wholesomeness 

2.  Contamination 

3.  Time/Temperature 

3.  Time/Temperature 

3.    Product  Handling 

4.  Cross-Contamination 

4.  Cross-Contamination 

5.  Packaging 

5.  Packaging 

Questionnaire  Design 

All  data  collection  instruments  were  pretested,  and  necessary  changes  were  made  in 
wording,  scaling,  or  ordering.  Correlation  meetings  were  held  with  the  data  collectors,  who 
were  thoroughly  trained  in  using  the  instruments  so  as  to  ensure  consistency  in  findings. 

Each  set  of  questions  in  the  data  collection  questionnaire  was  designed  to  measure 
whether  the  plant  1)  followed  an  established  procedure  (plant  monitoring);  2)  documented 
findings  (plant  records);  and  3)  took  corrective  action  (plant  response  to  process  deviations). 

For  example,  to  measure  the  plant's  control  of  the  foreign  material  and  organoleptic 
wholesomeness  of  raw  meat  received,  the  following  data  collection  questions  were  used: 

1 .  Does  plant  follow  established  procedures  in  performing  wholesomeness 
checks  on  incoming  meat/poultry  raw  materials? 

2.  Does  plant  document  its  findings  on  incoming  meat/poultry  raw  materials? 

3.  Does  plant  take  action  when  incoming  meat/poultry  raw  materials  do  not  meet 
specifications? 

The  Agency  conducted  verification  testing.  This  testing  was  done  using  a  fourth 
question  in  the  set  of  data  collection  tasks,  called  the  verification  task.  This  task  measured 
actual  adherence  by  the  plant  to  the  control  point's  requirements. 
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For  example,  to  measure  verification  of  the  control  point  foreign  material/ organoleptic 
wholesomeness  of  raw  meat  received,  the  fourth  question  in  the  data  collection  task  read: 

4.        Select  five  units  of  meat/poultry  raw  materials  at  receiving  and  determine  if 
''wholesome. 

The  data  collection  questions  were  applied  in  Phase  I  (Baseline)  and  again  in  Phase  III 
(Operational)  to  assess  plant  control  and  Agency  verification  before  and  after  HACCP 
implementation. 

Data  collection  task  and  frequency: 

Prior  to  the  study,  computer  modeling  was  conducted  to  simulate  scheduling  of  data 
collection  tasks  in  the  pilot  plants.  The  modeling  was  based  on  task  time  allocations 
developed  by  FSIS'  Industrial  Engineering  and  Ergonomics  Staff  and  assigned  frequencies  of 
three  times  per  week  for  the  scheduled  tasks.  As  a  result  of  the  modeling,  task  frequencies 
were  adjusted  to  fit  inspection  resource  allocations. 

The  frequency  for  the  scheduled  task  that  measured  adherence  to  established 
procedures  was  set  at  three  times  per  week.  This  provided  a  target  number  of  39  observations 
over  the  planned  13-week  study.  The  frequency  for  the  scheduled  verification  task  was  set  at 
three  times  per  week.  Assuming  a  13-week  study,  this  would  produce  39  observations. 

QUALITATIVE  DATA: 

Qualitative  data  were  collected  from  plant  records,  plant  personnel  and  inspection 
operations  personnel  to  provide  a  perspective  on  plant  management's  commitment  to  process 
control,  quality,  regulatory  compliance,  and  HACCP.  This  information  was  gathered  using 
various  survey  questionnaires.  Data  were  also  collected  to  assess  how  well  the  training  of 
Agency  and  plant  personnel  assisted  them  in  transitioning  into  a  HACCP  work  environment, 
survey/questionnaire/data  collection  instruments.  The  instruments  were  designed  and  pretested 
at  various  plants  using  Agency  headquarters'  staff  as  data  collectors. 

Due  to  resource  constraints,  the  number  of  plants  enlisted  into  the  study  as  well  as  the 
on-site  reviews  conducted  was  limited,  which  made  it  possible  for  the  data  to  be  biased  by 
day-to-day  operational  variability.  The  Agency  took  significant  steps  to  prevent  these  biases, 
including  extensive  pretesting  of  the  various  instruments,  training  of  data  collectors,  and 
holding  correlation  sessions  with  the  data  collectors  to  discuss  the  issues,  concerns,  and 
ambiguities  that  might  arise  from  the  instrument  design,  wording,  scaling,  and  scoring. 

The  remainder  of  this  section  discusses  the  objectives  and  designs  for  the  following 
qualitative  areas: 
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1 .  Management  commitment  to  manufacturing  processes,  quality,  and  regulatory 
compliance,  and 

2.  Industry  and  Agency  Training. 

s 

Plant  Management:  Manufacturing  Processes  and  Quality  and  Regulatory  Compliance 
Commitments  Methodology: 

In  an  effort  to  understand  the  possible  effects  of  HACCP  on  meat  and  poultry  plant's 
operating  environment,  eleven  qualitative  factors  were  identified  as  providing  an  overview  of  the 
processing  or  slaughter  plant  environment  and  provide  a  qualitative  assessment  of  the  overall  level 
of  regulatory  compliance  of  the  nine  pilot  plants.  The  factors  were  designed  to  measure  plant 
management's  commitment,  willingness,  and  capabilities  for  controlling  the  manufacturing 
process;  and  the  plant  management's  attitudes  toward  and  commitment  to  producing  quality  and 
regulatory  compliance- based  product,  especially  in  areas  that  impact  consumer  health  and  safety. 

FSIS  identified  11  qualitative  categories  which  are  displayed  in  Table  11.  The 
questionnaire  was  designed,  pretested,  and  modified  following  correlations  with  the  data 
collectors.  Besides  having  specific  criteria  for  each  of  the  major  categories,  data  collectors  also 
gave  narrative  descriptions.  The  most  significant  modification  to  the  questionnaire  was  the 
expansion  of  the  ordinal  rating  scales  (Likert-type  scale)  from  three  levels  to  five.  The  original 
high,  medium,  and  low  categories  were  expanded  and  defined  as  follows: 

1 .  High  -  a  factor  that  greatly  exceeds  regulatory  requirements; 

2.  High  medium  -  a  factor  that  meets  regulatory  requirements  with  some  additional 
effort  to  exceed  regulatory  requirements; 

3.  Medium  medium  -  a  factor  that  completely  meets  regulatory  requirements; 

4.  Low  medium  -  a  factor  that  just  barely  meets  the  minimum  regulatory 
requirements;  and 

5.  Low  -  a  factor  that  does  not  meet  regulatory  requirements. 
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TABLE  11:  QUALITATIVE  CATEGORIES  FOR  PLANT  MANAGEMENT: 
Manufacturing  Processes  and  Quality  and  Regulatory  Compliance  Commitments 
Methodology 


TTFM  NO 

OTIAT  TTATTVF  FACTORS 

1 

1  • 

Crpnpral  T-Tnnipk'ppnino 

VJIwllWlttl    1  luU^VIVW^Ulllg 

9 

{""r^nrlitinn  nf  fapilitif**: 

■3 
j . 

cnipiuycc  iiyj^iciic 

A 

employee  Saiiiuiry  praLiiueb 

J. 

employee  aiiuuucb  lowaru  lnspcLiiuii  icLjuircniciiib 

6. 

Employee  training 

7. 

Management  supervisory  attitudes 

8. 

Management  response  to  problems 

9. 

Management  strategies  to  prevent  contamination  hazards 

10. 

Product  quality 

11. 

Management  programs  and  systems 

The  original  medium  category  was  expanded  after  results  of  field  testing  the  questionnaire 
showed  most  plant  factors  at  that  level.  It  was  thought  that,  after  implementation  of  HACCP, 
many  of  these  factors  might  show  improvement,  but  still  qualify  only  for  a  medium  rating.  The 
expanded  scale  was  adopted  to  distinguish  a  high  medium  from  a  low  medium. 

Industry  and  Agency  Training: 

At  the  outset  of  the  HACCP  initiative,  FSIS  recognized  that  training  was  essential  to  the 
study's  success.  A  review  of  HACCP  literature  revealed  that  a  thorough  understanding  of 
HACCP  principles,  concepts,  and  techniques  is  necessary  for  individuals  who  develop  HACCP 
plans,  or  monitor  and  verify  a  plan's  operation.  This  includes  both  food  processors  and 
regulatory  personnel.  A  basic  understanding  of  HACCP  principles  is  important  for  food  handlers 
and  others  working  in  food  production,  as  well  as  an  understanding  of  the  importance  of  their 
individual  functions  within  the  HACCP  plan. 
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The  objective  of  the  HACCP  training  that  FSIS  conducted  at  the  participating  plants  was 
to  familiarize  both  plant  and  inspection  personnel  with  HACCP  principles.  The  Agency  sought 
to  give  plant  personnel  sufficient  understanding  to  begin  developing  their  plant-specific  HACCP 
plans.  Plants  were  encouraged  to  provide  additional  in-depth  training  to  employees  responsible 
for  developing  and  managing  the  HACCP  plans. 

The  training  took  place  at  the  participating  plants  (or  local  training  sites)  just  before  Phase 
II  of  the  HACCP  study.  The  primary  audience  for  this  training  was  inspection  personnel  assigned 
to  the  participating  plants.  Since  neither  the  data  collector  training  nor  the  initial  awareness 
training  was  assessed,  these  activities  are  not  addressed  in  this  report. 

The  HACCP  training  was  conducted  by  Agency  employees  who  had  helped  the  plants 
develop  their  specific  HACCP  plans  and  who  were  liaisons  with  the  plant  throughout  the  study. 
Each  training  session  lasted  three  days.  The  topics  included:  an  overview  of  the  HACCP  pilot 
study,  specific  in-depth  discussion  of  the  seven  HACCP  principles,  the  conversion  from  generic 
models  to  plant-specific  HACCP  plans,  and  in-depth  instruction  on  the  operation  of  the  PBIS  as 
modified  for  the  HACCP  plants.  Training  methods  included  lecture,  discussion,  and  hands-on 
workshops  to  demonstrate  practical  applications  of  HACCP  and  PBIS. 

The  training  was  assessed  to  determine  how  well  it  functioned,  to  gain  insights  on  the 
training  needs  of  inspectors  working  in  a  HACCP-based  inspection  system,  to  gain  insights  on 
the  content  needed  in  HACCP  training,  to  begin  consideration  of  the  target  population  for 
HACCP  training,  and  to  learn  about  the  best  formats  for  delivering  such  a  training  program. 

The  immediate  aim  of  the  training  assessment  was  to  provide  feedback  to  the  Agency  to 
improve  future  training  sessions  by  assessing  the  HACCP  training  needs  of  FSIS  inspection 
personnel  and  gauging  the  effectiveness  of  the  delivery  method. 

Data  Collection: 

The  data  collection  methodology  used  in  the  training  assessment  consisted  of:  1)  direct 
observation  of  FSIS  training  of  its  employees  and  industry  personnel  at  training  sites;  2) 
administration  of  a  reaction  survey  questionnaire  to  trainees  immediately  after  training;  3) 
observation  of  FSIS  employees'  performances  during  Phase  II  (implementation)  and  Phase  III 
(operational)  of  the  pilots;  4)  informal  interviews  with  FSIS  employees;  and  5)  review  of  the 
training  materials. 

Agency  evaluators  observed  one  training  session  from  each  of  the  product/process 
categories.  Evaluators'  attendance  at  training  sessions  was  scheduled  so  that  each  pair  of  FSIS 
trainers  could  be  observed  leading  a  session.  Evaluators  reviewed  lesson  plans,  visual  aids,  and 
other  materials  to  determine  their  usefulness  in  meeting  the  training  objectives.  Participants  at 
all  training  sessions  were  asked  to  fill  out  an  evaluation  sheet.  Participants  included  FSIS 
personnel  (inspectors,  relief  inspectors,  circuit  supervisors,  area  officials,  regional  officials,  and 
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HACCP  data  collectors)  and  plant  personnel,  plant  managers,  production  supervisors,  and  quality 
control/quality  assurance  staff.  All  data  to  assess  the  training  were  forwarded  to  FSIS 
headquarters  for  analysis. 
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CHAPTER  III 


ANALYSIS  FOR  REFRIGERATED  FOODS,  COOKED  SAUSAGES 
AND  POULTRY  SLAUGHTER  HACCP  SYSTEMS 
PILOT  TESTING  AT  NINE  VOLUNTEER  PLANTS 


INTRODUCTION: 


Detailed  discussions  and  analyses  of  pilot  test  results  of  plant-specific  HACCP 
plans  for  Refrigerated  Foods,  Cooked  Sausages,  and  Poultry  Slaughter  at  nine  volunteer 
plants  will  be  presented  for  National  Profiles,  HACCP  Model  Checklist,  Quantitative  Plant 
Data,  and  Qualitative  Data.  To  keep  the  confidentiality  of  the  participating  plants' 
identities,  they  will  be  identified  as  1,  2,  or  3  for  their  respective  product  areas  (i.e.,  RF  1, 
RF  2,  RF  3;  CS  1,  CS  2,  CS  3;  PS  1,  PS  2  and  PS  3). 


NATIONAL  PROFILES:  IDENTIFYING  REFRIGERATED  FOODS  PLANTS, 
COOKED  SAUSAGE  PLANTS,  AND  POULTRY  SLAUGHTER  PLANTS. 

To  present  an  overview  of  the  general  plant  population  from  which  the  plants 
volunteered,  FSIS  collected  and  analyzed  information  from  all  meat  and  poultry  plants 
producing  ready-to-eat  refrigerated  foods,  cooked  sausages,  and  young  chickens  (poultry 
slaughter).  The  1992  profile  data  were  collected  from  the  Agency's  Performance  Based 
Inspection  System  (PBIS)  and  Animal  Disposition  Reporting  System  (ADRS)  data  bases  to 
determine  the  general  characteristics  of  plants  by  size  (as  measured  by  square  feet  of  plant 
space)  and  production  volume. 

USDA  Plant  Population: 

FSIS  inspects  approximately  6,200  meat  and  poultry  establishments  (see  Table  12). 
Over  1,600  are  processing  plants  only,  while  there  were  less  than  400  slaughter  plants,  and 
more  than  1,300  combination  plants  (a  plant  that  slaughters  and  performs  processing  is  a 
combination  plant/establishment). 
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TABLE  12:  NUMBER  AND  TYPE  OF  PLANTS  UNDER  FEDERAL  INSPECTION 


TYPE 

NUMBER 

Processing 

Slaughter 

Combination 

4,557 
378 
1,345 

TOTAL 

6,280 

By  reviewing  PBIS  and  other  plant  survey  data,  FSIS  identified  322  Refrigerated 
Food  plants  and  1,004  Cooked  Sausage  plants  for  study.1  Using  the  ADRS  data  base, 
FSIS  identified  225  poultry  plants  that  slaughtered  young  chickens. 

Table  13  identifies  plants  as  Processing,  Red  Meat  Slaughter,  and  Poultry  Slaughter 
Operations.  When  performing  the  profile  analysis,  it  is  important  to  differentiate  between 
a  plant/establishment  and  the  operations  a  plant  conducts.  A  plant  may  perform  multiple 
types  of  operations.  The  term  operations  refers  to  the  number  of  different  species 
slaughtered  or  processing  operations  being  performed  by  a  plant.  A  slaughter  plant  may 
slaughter  beef  and  swine  or  chicken  and  turkey.  A  processing  plant  may  produce  cooked 
sausages  and  a  ready-to-eat  refrigerated  food  product.  Processing  represents  nearly  two 
thirds  of  all  federally-inspected  plants,  while  poultry  slaughter  represents  five  percent  (See 
Chart  1).  Table  14  shows  that  the  Refrigerated  Foods  plants  are  around  seven  percent 
(322  plants)  and  Cooked  Sausage  plants  about  16  percent  (1,004  plants)  of  the  6,253 
processing  operations  (See  Chart  2). 

TABLE  13:  NUMBER  OF  FEDERALLY-INSPECTED  OPERATIONS  FOR 
PROCESSING,  RED  MEAT  SLAUGHTER  AND  POULTRY  SLAUGHTER 


TYPE  OF  PLANT 

NUMBER 

PERCENT  OF 
TOTAL 

PROCESSING 

6,253 

65.75 

RED  MEAT  SLAUGHTER 

2.757 

29.0 

POULTRY  SLAUGHTER 

509 

5.3 

TOTAL 

9,519 

100.0 

Plant  number  reporting  PBIS  data  may  vary  due  to  multiple  shifts. 
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TABLE  14:  POPULATION  OF  PROCESSING  OPERATIONS  BY 
REFRIGERATED  FOODS,  COOKED  SAUSAGES  AND  ALL  OTHERS 


PROCESSING  PLANTS 

PERCENT  OF 

(io-plant  numbers) 

TOTAL 

REFRIGERATED  FOODS 

322 

6  9 

COOKED  SAUSAGE 

1,004 

16.0 

OTHER 

4,821 

77.1 

TOTAL 

6,253 

100.0 

Table  15  displays  the  composition  of  Red  Meat  Slaughter  and  Poultry  Slaughter 
operations.  Of  the  2,757  Red  Meat  Slaughter  operations  (see  Chart  3),  Beef  Slaughter 
represents  approximately  two  in  every  five  operations  (37.3  percent)  while  Swine 
Slaughter  is  around  one  in  every  three  operations  (34.2  percent).  For  Poultry  Slaughter 
(See  Chart  4),  Chicken  Slaughter  dominates,  representing  nearly  three  in  every  five 
operations  (59.5  percent). 

TABLE  15:  RED  MEAT  AND  POULTRY  SLAUGHTER  PLANTS  BY  SPECIFIC 
SPECKS  COMPOSITION 


SLAUGHTI 

•R  PLANTS 

Red  Meat 

Percent  of  Total 

Poultry 

Percent  of  Total 

Beef 

1,028 

37.3 

Chicken 

303 

59.5 

Swine 

942 

34.2 

Turkey 

156 

30.6 

Goats,  Sheep,  Lamb 

743 

26.9 

Ducks 

26 

5.1 

Equine 

12 

0.4 

Geese 

9 

1.8 

Other 

32 

1.16 

Rabbits 

4 

0.8 

Others  (guineas, 
squab,  pigeons) 

11 

2.2 

Total* 

2,757 

Total* 

509 

*  Total  compounded  thus  does  not  represent  absolute  number  of  plants.  For  example,  one  plant  may  slaughter  both  beef  and 
swine. 
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Characteristics  used  to  profile  processing  operations  were:  plant  volume  (pounds 
produced  annually)  and  size  (square  footage).  Table  16  displays  the  distribution  of 
processing  operations  by  square  footage.  Of  the  6,253  processing  operations,  41  percent 
were  medium  volume  plants  (producing  between  240,000  and  4,000,000  pounds  of 
product  annually  [See  Chart  5]),  while  33  percent  were  high  volume  producers  (more 
than  4,000,000  pounds).  Low  volume  producers  (less  than  240,000  pounds  annually) 
represent  one  in  every  four  (26  percent).  With  respect  to  square  footage,  small  plants 
(having  less  than  12,000  square  feed  of  plant  area)  dominated;  64  percent  of  all 
processing  plants  were  small.  Nine  percent  of  the  plants  were  classified  as  large  (i.e., 
more  than  80,000  square  feet  of  plant  area. 

TABLE  16:  CLASSIFICATIONS  FOR  PRODUCTION  AND  SQUARE  FOOTAGE 
INTERVALS  WITH  THE  NUMBER  AND  PERCENTAGE  OF  PLANTS  IN  EACH 
INTERVAL 


Category 

Interval 

Number  of  Plants 

Percent  of  Total 

Annual  Volume  in  Pounds 

Pounds 

Low 

<  240,000 

1,605 

26 

Medium 

240,000  -  4,000,000 

2,570 

41 

High 

>  4,000,000 

2,078 

33 

Total4 

6,253 

100 

Square  Footage** 

Interval 

Square  Feet 

Small 

<  12,000 

3.901 

64 

Medium 

12,000  -  80,000 

1.639 

27 

Large 

>  80.000 

547 

9 

Total 

6,087 

100 

Production  of  Official  Establishment  (POOE)  Data  for  1992 

Square  footage  data  is  reported  by  CORE.  Data  -were  not  reported  for  166  establishments. 


The  Production  of  Official  Establishment  (POOE)  data  represents  the  annual  volume 
reported  by  5,411  of  the  6,253  processing  plants  in  1992.  Chart  6  shows  that  high  volume 
producers  (i.e.  over  4,000,000  pounds  annually)  accounted  for  95  percent  of  the  64.7  billion 
pounds,  with  average  annual  production  for  the  1,355  plants  reporting  being  nearly  44  million 
pounds.  The  total  volume  of  1 ,704  small  volume  producers  represented  1  percent  of  the  total 
annual  production,  with  the  average  annual  production  of  a  small  plant  being  nearly  357,000 
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pounds.  Chart  7,  which  portrays  the  Refrigerated  Foods,  Cooked  Sausage,  and  all  other 
processing  plant  data,  shows  that  Cooked  Sausage  plants  (16.2  percent  of  the  plants) 
represented  about  20  percent  of  the  annual  production. 

For  Red  Meat  and  Poultry  Slaughter,  the  item  of  interest  is  the  number  of  head  or 
birds  slaughtered  annually.  Table  17  displays  the  red  meat  total  slaughter  and  poultry 
slaughter  figures.  In  1 992  nearly  7  billion  animals  were  slaughtered.  The  poultry  slaughter 
represented  nearly  6.9  billion  birds.  Chicken  slaughter  represents  nearly  95  percent  of  all 
poultry  slaughter.  Red  meat  slaughter  accounted  for  nearly  127  million  head  in  1992. 

Refrigerated  Foods  Plants: 

Examination  of  the  Refrigerated  Foods  plant  population  for  the  volume  characteristic, 
Chart  8  indicated  a  distribution  similar  to  the  general  plant  population  (see  Chart  5). 
However,  when  using  the  criterion  of  size  (i.e.,  square  footage),  only  half  of  the  Refrigerated 
Foods  plants  are  in  the  small  range,  compared  to  64  percent  of  all  plants.  Nearly  three  in 
every  eight  Refrigerated  Foods  plants  (38  percent)  were  classified  as  medium  in  size,  versus 
a  little  over  one  in  four  (27  percent)  for  the  general  plant  population.  The  category  of  large 
size  plants  showed  about  the  same  for  the  Refrigerated  Foods  plants  as  for  the  general  plant 
population  (11  percent  and  9  percent,  respectively).  All  three  volunteer  plants  were  high 
volume  producers;  there  was  one  plant  in  each  size  category. 

Cooked  Sausage  Plants: 

As  seen  in  Charts  5  and  9,  comparing  Cooked  Sausage  plants  to  the  general  plant 
population  shows  more  Cooked  Sausage  plants  in  the  medium  category  for  annual  production 
volume  than  the  general  population  (47  percent  and  41  percent,  respectively)  with  the  shift 
coming  from  a  lower  percentage  of  high  volume  producers  (28  percent  and  33  percent, 
respectively).  When  looking  at  plant  size,  the  Cooked  Sausage  plants  are  similar  to  the 
general  population.  There  is  a  slight  shift  in  the  percentage  of  medium  sized  firms,  with  the 
difference  coming  from  a  lower  percentage  of  small  plants  in  the  Cooked  Sausage  plants  (59 
percent,  compared  to  64  percent  for  the  general  population).  The  cooked  sausage  plants' 
annual  production  volumes  classified  CS  1  and  CS  3  as  high  volume  plants. 

Poultry  Slaughter  Plants: 

For  Fiscal  Year  1992,  ADRS  data  showed  that  more  than  6.852  billion  poultry  were 
slaughtered  (see  Table  1 7)  at  320  poultry  slaughter  plants.  Young  chickens  were  slaughtered 
in  225  of  the  plants  (70.3  percent  of  the  320  plants)  and  dominated  the  total  poultry  slaughter 
volume,  with  more  than  6.368  billion  birds  (92.9  percent).  Table  17  shows  the  classification, 
number,  and  percentage  of  poultry  plants  in  each  category,  as  well  as  the  percentage  of  total 
poultry  slaughter  volume. 
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TABLE  17:  RED  MEAT  AND  POULTRY  SLAUGHTER  BY  CLASS  FOR  USDA 
INSPECTED  SLAUGHTER  PLANT  OPERATIONS  -  FY  '92 


Head  Slaughtered 

rClLCZjL  Ol 

Slaughter 

1  ^1 UII1LKTT  UI 

Operations 

P^rr#*nf  nf  thp 
I  CICCIJ  L  UI  LUC 

2757  Plants 

Red  Meat 

Cattle 

J  —  ,  1  i     ,    o J 

25.3 

1028 

37.3 

Swine 

89,210,132 

70.3 

942 

34.2 

Sheep/Goat 

5,354,043 

4.2 

743 

26.9 

Other  Red  Meat 

247,273 

0.2 

44 

1.6 

Total 

126,923,811 

100 

H^aH  ^  1  an trh fprprl 

r  ci  tcui  ui 

Slaughter 

Numnpr  />f 

Plants 

Pprrpnt  nf  f hp 

320  Plants 

Young  Chickens 

6,368,360,156 

92.9% 

225 

70.3% 

Light  Fowl 

129,530,164 

1.99% 

38 

11.9% 

Fryer-Roaster  Turkeys 

1,403,436 

0.02% 

4 

1.3% 

Young  Turkeys 

274,092,153 

4.0% 

90 

28.1% 

Old  Breeder  Turkeys 

2,394,944 

0.0% 

34 

10.6% 

Ducks 

18,027,590 

0.3% 

26 

8.1% 

Geese 

161,449 

0.0% 

9 

2.8% 

Rabbits 

431,975 

0.0% 

4 

1.3% 

Others  (Guineas,  Squabs  etc.) 

4,442,675 

0.1% 

11 

3.4% 

Capons 

288,729 

0.0% 

10 

3.1% 

Heavy  Fowl 

51,309,759 

0.7% 

30 

9.4% 

Young  Breeder  Turkeys 

1,709,070 

0.0% 

28 

8.8% 

Total 

6,852,152,100 

100.0% 
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Table  1 8  presents  the  same  classifications  for  young  chicken  slaughter.  Low  volume 
slaughterers  (less  than  10  million)  represented  almost  30  percent  of  the  plants,  but  less  than 
2  percent  of  the  total  birds  slaughtered,  while  the  high  volume  producers  (more  than  50 
million  birds)  represented  approximately  15  percent  of  the  plants,  but  nearly  35  percent  of 
the  slaughter  volume.  PS  1  and  PS  3  were  high  volume  slaughter  plants,  while  PS  2  was 
a  medium  volume  plant.  Chart  1 0  presents  the  poultry  slaughter  data  by  low,  medium,  and 
high  volume  classifications.  The  average  annual  slaughter  for  low,  medium,  and  high  volume 
producers  was  1.5,  31.6,  and  86.0  million  birds,  respectively. 

TABLE  18:  CLASSIFICATION  AND  NUMBER  VOLUME  OF  YOUNG  CHICKEN 
SLAUGHTER  PLANTS  -  FY  '92 


Classification 

Range 

Number  of 
Plants 

%  of  Plants 

%  of  Young 

Chickens 
Slaughtered 

Low 

<  10,000,000 

66 

29.3 

1.8 

Medium 

10,000,001  - 
50,000,000 

125 

55.6 

63.5 

High 

>  50,000,000 

34 

15.1 

34.7 

Total 

225 

100.0 

100.0 

Table  19  displays  the  number  of  young  chickens  slaughtered,  number  of  condemned 
chickens,  and  the  condemnation  rate  at  all  young  chicken  slaughter  operations,  PS  1,  PS  2, 
and  PS  3.  FY  92  ADRS  data  showed  6.4  billion  young  chickens  were  slaughtered  at  225 
young  chicken  slaughter  operations.  The  average  slaughter  was  28.3  million  young  chickens 
per  plant.  All  three  plants  (PS  1,  PS  2,  and  PS  3)  were  above  the  average.  PS  2  slaughtered 
over  30  million  while  PS  1  and  PS  3  slaughtered  76.7  and  62.1  million,  respectively.  The 
total  number  of  young  chickens  condemned  in  FY  1992  was  61,548,308  which  translates  into 
a  national  condemnation  rate  of  0.97  percent.  As  seen  in  Table  19,  PS  1,  PS  2,  and  PS  3 
rates  were  1.08  percent,  0.81  percent,  and  0.52  percent,  respectively.  PS  1  (1.08  percent)  was 
slightly  above  the  national  rate  (0.97  percent),  while  PS  3  (0.52)  was  almost  haJf  of  the 
national  rate. 
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TABLE  19:  NUMBER  OF  YOUNG  CHICKENS  CONDEMNED  AND 
CONDEMNATION  RATE  FOR  PS  1,  PS  2,  AND  PS  3 


PLANT 

NUMBER  SLAUGHTER 

NUMBER  CONDEMNED 

PERCENT  OF  TOTAL 

PS-1 

76,714,340 

824,942 

1.08 

PS-2 

30,303,122 

244,313 

0.81 

PS-3 

62,086,896 

323,810 

0.52 

ALL  PLANTS 

6,368,360,156 

61,548,308 

0.97 

HACCP  MODEL  CHECKLIST 

This  section  discusses  the  conformance  of  the  nine  plant-specific  plans  and  the  three 
generic  model  HACCP  plans  to  the  seven  principles  recommended  by  the  NACMCF.  More 
detail  on  the  checklist  methodology  appears  in  Chapter  2.  Table  20  shows  the  summary  of 
the  scoring  and  point  deductions  for  all  nine  volunteer  plant  plans. 

Refrigerated  Foods: 

Of  the  50  questions  in  the  checklist,  the  generic  Refrigerated  Foods  HACCP  Plan 
addressed  38,  with  12  found  to  be  not  applicable  (see  Table  21).  The  Refrigerated  Foods 
HACCP  model  was  found  to  be  in  100  percent  conformance  with  the  NACMCF 
recommendations.  All  three  Refrigerated  Foods  plants'  specific  plans  addressed  all  50 
questions.  RF  3  was  in  98  percent  conformance  with  NACMCF  recommendations,  while  RF 
l's  and  RF  2's  plans  were  each  in  97  percent  conformance.  None  of  the  three  Refrigerated 
Foods  plants'  plans  adequately  addressed  (i.e.,  specifically  listed)  the  microbiological  hazards 
for  the  Hazard  Assessment  area  in  the  checklist,  as  shown  in  Table  20.  Additionally,  none 
of  their  plans  fully  addressed  the  basis  for  the  frequency  of  monitoring  under  establishment 
of  monitoring  requirements. 
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TABLE  20:  HACCP  MODEL  CHECKLIST  SCORING  SUMMARY  AND  POINT 
DEDUCTIONS  FOR  THE  REFRIGERATED  FOODS,  COOKED  SAUSAGES,  AND 
POULTRY  SLAUGHTER  VOLUNTEER  PLANTS 


PLAN 

POSSIBLE 
POINTS 

POINTS 
DEDUCTED 

QUESTIONS  NOT  FULLY  ADDRESSED* 

RF 

MODEL 

100 

0 

1 

100 

3 

B5 

E2 

E4 

2 

100 

3 

B5 

E2 

E4 

3 

100 

2 

B5 

E2 

CS 

MODEL 

100 

1 

H3 

1 

100 

2 

E2 

E4 

2 

100 

2 

E2 

E4 

3 

100 

0 

PS 

MODEL 

100 

2 

E6 

H4b 

1 

100 

2 

E2 

E4 

2 

100 

2 

E6 

H4b 

3 

100 

2 

E6 

H4b 

TOTALS 

1200 

21 

3 

6 

5 

1 

3 

3 
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TABLE  21:  CHECKLIST  RESPONSE  TOTALS  AND  SCORING  SUMMARY  FOR 
REFRIGERATED  FOODS  HACCP  MODEL  PLANS 


Summary 'of  responses  to  questions: 


PLAN 

TOTAL  NUMBER 
OF  QUESTIONS 

NUMBER 
ADDRESSED  IN 
PLAN 

NUMBER  NOT 
ADDRESSED  IN 
PLAN 

NUMBER  NOT 

FULLY 
APPLICABLE 

WORKSHOP 
MODEL 

50 

38 

0 

12 

PLANT  NUMBER  1 

50 

50 

0 

0 

PLANT  NUMBER  2 

50 

50 

0 

0 

PLANT  NUMBER  3 

50 

50 

0 

0 

Summary  of  scores: 


PLAN 

TOTAL 
POSSIBLE 
SCORE 

SCORE  OF 

"YES" 
RESPONSES 

SCORE  OF 

"NO" 
RESPONSES 

SCORE  OF 
"NOT  FULLY 
APPLICABLE 
"  RESPONSES 

PERCENT 
CONFORMANCE 
TO  NACMCF 
RECOMMENDATIO 
NS 

WORKSHOP 
MODEL 

100 

77 

0 

23 

100% 

PLANT  NUMBER  1 

100 

97 

0 

0 

97% 

PLANT  NUMBER  2 

100 

97 

0 

0 

97% 

PLANT  NUMBER  3 

100 

98 

0 

0 

98% 

Cooked  Sausage: 

Table  22  presents  the  summary  of  responses  and  scores  for  the  Cooked  Sausage  topic 
plants,  while  Table  20  includes  the  scoring  and  point  deductions.  As  with  the  Refrigerated 
Foods  generic  HACCP  plan,  the  Cooked  Sausage  Generic  HACCP  plan  addressed  38  out  of 
the  50  questions,  with  12  found  to  be  not  applicable. 

However,  the  Cooked  Sausage  generic  plan  did  not  receive  a  100  percent  score 
because  it  did  not  address  the  frequency  of  verification  inspection.  CS  3's  plan  fully 
conformed  to  the  NACMCF  recommendations,  for  a  score  of  100  percent.  CS  l's  and  CS 
2's  plans  scored  98  percent;  neither  plan  addressed  the  frequency  of  monitoring  nor  included 
the  documentation  relative  to  the  effectiveness  of  established  cooking,  cooling,  etc., 
procedures. 


52 


TABLE  22:  CHECKLIST  RESPONSE  TOTALS  AND  SCORING  SUMMARY  FOR 
COOKED  SAUSAGE  HACCP  MODEL  PLANS 


Summary  "t>f  responses  to  questions: 


PLAN 

TOTAL  NUMBER 
OF  QUESTIONS 

NUMBER 
ADDRESSED  IN 
PLAN 

NUMBER  NOT 
ADDRESSED  IN 
PLAN 

NUMBER  NOT 

FULLY 
APPLICABLE 

WORKSHOP 
MODEL 

50 

38 

0 

12 

PLANT  NUMBER  1 

50 

50 

0 

0 

PLANT  NUMBER  2 

50 

50 

0 

0 

PLANT  NUMBER  3 

50 

50 

0 

0 

Summary  of  scores: 


PLAN 

TOTAL 
POSSIBLE 
SCORE 

SCORE  OF 

"YES" 
RESPONSES 

SCORE  OF 

"NO" 
RESPONSES 

SCORE  OF 
"NOT  FULLY 
APPLICABLE 
"  RESPONSES 

PERCENT 
CONFORMANCE 
TO  NACMCF 
RECOMMENDATTO 
NS 

WORKSHOP 
MODEL 

100 

76 

0 

23 

99% 

PLANT  NUMBER  1 

100 

98 

0 

0 

98% 

PLANT  NUMBER  2 

100 

98 

0 

0 

98% 

PLANT  NUMBER  3 

100 

100 

0 

0 

100% 

Poultry  Slaughter: 

The  checklist  response  totals  and  scoring  summary  for  Poultry  Slaughter  are  displayed 
in  Table  23;  while  the  scoring  and  point  deductions  appear  in  Table  20.  The  Poultry 
Slaughter  HACCP  generic  plan  addressed  38  questions,  with  12  found  not  to  be  fully 
applicable.  The  overall  score  for  the  generic  plan  was  98  percent  because  two  questions  were 
not  fully  addressed.  The  Poultry  Slaughter  generic  plan  did  not  specifically  state  the  need  for 
designating  responsible  individuals  and  their  signatures  for  the  monitoring  data.  It  also  failed 
to  specifically  state  the  need  for  designating  responsible  individuals  and  their  signatures  for 
CCP  monitoring. 

PS  l's  plant-specific  plan  showed  deficiencies  similar  to  those  of  RF  2  and  RF  3  and 
CS  1  and  CS  2.  PS  1  's  plan  did  not  specifically  address  the  need  for  designating  responsible 
individuals  and  signatures  for  monitoring  data  or  the  frequency  of  collection.  PS  2's  and  PS 
3's  plans  showed  the  same  deficiency  as  the  generic  Poultry  Slaughter  plan  by  not  designating 
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responsible  individuals  and  their  signatures  for  CCP  monitoring.  Neither  PS  2's  nor  PS  3's 
plans  specifically  mentioned  the  need  for  designating  individuals  and  their  signatures  for 
monitoring  data. 

TABLE  23:  CHECKLIST  RESPONSE  TOTALS  AND  SCORING  SUMMARY 
CHECKLIST  FOR  POULTRY  SLAUGHTER  HACCP  MODEL  PLANS 


Summary  of  responses  to  questions: 


PLAN 

TOTAL  NUMBER 
OF  QUESTIONS 

NUMBER 
ADDRESSED  IN 
PLAN 

NUMBER  NOT 
ADDRESSED  IN 
PLAN 

NUMBER  NOT 

FULLY 
APPLICABLE 

WORKSHOP 
MODEL 

50 

38 

0 

12 

PLANT  NUMBER  1 

50 

50 

0 

0 

PLANT  NUMBER  2 

50 

50 

0 

0 

PLANT  NUMBER  3 

50 

50 

0 

0 

Summary  of  scores: 


PLAN 

TOTAL 
POSSIBLE 
SCORE 

SCORE  OF 

■yes: 

RESPONSES 

SCORE  OF 

'NO" 
RESPONSES 

SCORE  OF 
"NOT  FULLY 
APPLICABLE 
"  RESPONSES 

PERCENT 
CONFORMANCE 
TO  NACMCF 
RECOMMEND  ATIO 
NS 

WORKSHOP 
MODEL 

100 

75 

0 

23 

98% 

PLANT  NUMBER  1 

100 

98 

0 

0 

98% 

PLANT  NUMBER  2 

100 

98 

0 

0 

98% 

PLANT  NUMBER  3 

100 

98 

0 

0 

98% 

In  summary,  the  major  deficiencies  revealed  by  the  HACCP  model  checklist  were  that 
six  of  the  nine  plant's  plans  lacked  specificity  in  their  monitoring  requirement  procedures  for 
continuous  monitoring,  or  the  plans  did  not  specifically  state  the  statistical  based  frequency 
of  the  non-continuous  monitoring  requirements,  while  five  of  the  nine  plans  (two  Refrigerated 
Foods,  two  Cooked  Sausage  and  one  Poultry  Slaughter)  did  not  specifically  mention  the  need 
for  designating  responsible  individuals  and  their  signatures  for  monitoring  data. 
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ANALYSIS  OF  MICROBIOLOGICAL,  CHEMICAL,  AND  PHYSICAL  QUANTITATIVE 
DATA: 


As  discussed  under  Methodology,  Chapter  2,  testing  for  microbiological  chemical,  and 
physical  factors  was  conducted  to  ascertain  the  degree  of  process  control  that  each  of  the  nine 
volunteer  plants  exercised.  Since  chemistry  factors  were  used  to  assess  how  microbial  growth 
was  affected,  the  analyses  of  microbiological  and  chemistry  factors  will  be  discussed  together 
for  each  of  the  three  product  types.  This  section  will  first  discuss  the  results  of  the  baseline 
and  operational  phases  (Phases  I  and  III)  testing  for  the  microbiological  and  chemical  factors, 
and  then  the  results  of  the  physical  factors  testing.  Chapter  2  (Methodology)  provides  a 
discussion  on  the  statistical  methods  used  for  analyzing  the  microbiological  data. 

Microbiological  and  Chemical  Factors  Analyses: 

Refrigerated  Foods  Plants: 

The  quantitative  data  for  the  three  volunteer  Refrigerated  Foods  plants  included  seven 
microbiology  factors  and  no  chemistry  factors.  The  seven  microbiology  factors  are  listed  in 
Chapter  II. 

Samples  were  collected  for  the  first  six  of  these  factors  at  three  different  points  in 
processing:  (1)  before  cooking;  (2)  after  cooking  and  chilling;  and  (3)  from  final  packaged 
product.  Results  for  these  factors  were  reported  in  counts  per  gram.  Swabs  for  Listeria 
species  tests  were  taken  from  four  different  equipment  surfaces  on  an  alternating  basis. 
Results  for  Listeria  tests  were  reported  as  simply  positive  or  negative  (not  quantified). 

The  sampling  sites  were  purposely  selected  to  bracket  critical  control  points  in  the 
production  of  refrigerated  foods.  These  critical  control  points  are  those  related  to  cooking, 
chilling,  and  post-cooking  handling  of  the  product.  The  cooking  process  is  the  only  critical 
control  point  in  the  production  of  this  product  where  an  appreciable  reduction  of  microbial 
levels  can  be  expected.  The  sampling  sites  are  depicted  in  Tables  24,  26  and  28  in  relation 
to  both  the  critical  control  points  and  other  steps  in  the  production  process.  The  mean 
microbiological  levels  for  each  of  the  microbiological  factors  at  each  of  the  sampling  sites  are 
shown  in  the  same  tables.  The  values  in  the  tables  for  points  prior  to  cooking  show  the 
microbiological  condition  of  the  raw  material.  The  values  immediately  after  cooking  and 
chilling  indicate  the  effectiveness  of  the  cooking  process  in  reducing  microbial  levels  and  the 
effectiveness  of  the  chilling  process  in  retarding  multiplication  of  any  survivors.  The  final 
values  in  the  tables  indicate  the  degree  of  sanitation  practiced  during  the  steps  of  the  process 
resulting  in  the  final  packaged  product.  In  these  tables,  as  indicated  in  the  notes  under  the 
tables,  a  value  of  one  was  assigned  to  samples  which  were  negative  to  facilitate  the 
calculation  of  the  geometric  mean. 
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Tables  25,  27  and  29  present  the  percent  of  samples  testing  positive  for  each  of  the 
microbiological  factors  and  the  log  average  level  for  each. 

The  plan  to  collect  sample  sets  on  30  to  50  different  days  during  both  the  baseline  and 
operational  phases  (Phases  I  and  III)  was  fulfilled  for  two  of  the  three  Refrigerated  Foods 
plants  and  nearly  fulfilled  in  the  other  plant  (only  27  daily  samples  were  collected  during 
Phase  III  in  that  plant).  The  results  were  analyzed  to  check  each  plant's  ability  to  control  its 
process  and  to  compare  characteristics  of  product  produced  in  Phases  I  and  III.  Results  are 
discussed  separately  for  each  plant. 

Results  for  microbiology  factors  from  Phase  I  were  shared  with  the  plants  at  the 
beginning  of  Phase  II,  and  results  from  Phase  II  were  shared  as  they  became  available. 
Because  of  the  5-to-6  week  lag  between  sample  collection  and  feedback  of  results,  the  data 
were  not  useful  for  monitoring  process  control,  but  were  of  retrospective  value  for  assessment 
purposes. 

In  summary,  the  HACCP  microbiological  factors  in  RF  1  showed  little,  if  any, 
practical  difference  between  Phase  I  and  Phase  III  microbiologically.  The  microbiological 
profile  of  product  produced  during  both  phases  is  indicative  of  good  manufacturing 
procedures. 

Refrigerated  Foods  plant  1  (RF  1): 

Table  24  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at  the 
different  sampling  points  for  RF  1  during  Phases  I  and  III.  It  relates  the  sampling  points  to 
the  generic  HACCP  Refrigerated  Foods  model  and  demonstrates  the  change  in  the  microbial 
profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  of  incoming  raw 
material  were  low.  The  cooking  process  further  reduced  the  levels  of  microorganisms  —  in 
many  cases,  to  below  the  level  of  detection.  The  levels  of  microorganisms  in  the  final 
product  were  slightly  higher,  reflecting  post-cooking  handling  of  the  product;  however,  the 
overall  microbiological  profile  of  the  final  product  in  both  Phase  I  and  Phase  III  is  indicative 
of  excellent  processing  procedures.  Because  of  these  low  levels,  there  is  no  practical 
difference  between  the  microbial  profiles  of  product  sampled  during  Phase  I  and  product 
sampled  during  Phase  III. 

Table  25  gives  more  detailed  summary  statistics  for  the  microbiology  factors  at  the 
three  sampling  points  during  Phases  I  and  III.  There  are  a  greater  number  of  samples  for 
final  product  because  duplicate  final  product  samples  were  collected  each  sampling  day  in  this 
plant.  Only  APC  @  35°C  and  APC  @  20°C  showed  appreciable  percent  positives  in  the  final 
product  (see  Pet  values).  Figures  1  and  2  show  the  distribution  of  APC  @  35  and  20°C 
respectively  for  RF  1 .  As  seen,  the  values  drop  significantly  from  the  raw  product  process 
to  final  product  both  in  Phase  I  and  Phase  III.  The  slight  increase  in  samples  testing  positive 
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TABLE  24:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  III  REFRIGERATED  FOODS  PLANT  1 


Process  Step  APC  @  35°C       APC  @  20°C        Coliforms  £  colt  S.  aureus  GFAs 

Receiving 

I 

Storage 


Preparation 

I 


Cooking* 
Chilling* 


Assembly* 

I 

Flush  (Gas)* 

I 

Seal  Package 
I 

Inspect  Package" 

I 

Labeling,  etc.* 
I 

Chilling* 


Storage* 


—  1,280  2,730  22  1  3  2 

|  1,880  3,640  23  1  2  5 


1  11111 
1  1  1111 


2  2  1111 

2  11111 


Notes: 

1.  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  All  values  are  reported  on  a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=10)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 


57 


Std 

■>f  IT 
\C  UX 

vn  "0- 

U-l  xD 

o 
o 

'roduct 

> 

< 

rx.  xo 

rx  vO 

o 

o 

Final  F 

G 

CU 

17.6 
14  6 

19.1 
10.4 

—  o 
— "  o 

o  o 
d  d 

o  o 
d  d 

in  o 
,  d  d 

E 

3 

Z 

OO  sC 
OO  ox 

oo  ^ 

OO  ox 

OO 

OO  o> 

OO 

00  ox 

OO  >r> 

00  ox 

OO  %£> 

oo  ox 

-u 

■»»■  i-v 

m  sc> 

Product 

> 

< 

u-i  in 

OO  IN 

<* 

Cooked 

e 

Cu, 

hx  in 
od  T 

m  ~- 

V  in 

—  o 

—  O 

o  o 
d  d 

O  O 

d  d 

o  o 
d  d 

E 

3 

2 

IN  OO 
OX  TT 

IN  OO 
OX  -*J- 

<N  00 
ox  ■»?■ 

IN  OS 
OX 

(XI  oo 
ox  *r 

IN  OO 

ox  -*r 

-o 

«/? 

>D  l"X 
-t  ♦ 

IN  IN 
vD  vD 

oo  fxi 
"<r  ux. 

o  o 
■*■  o 

— 

*n  in 

<-  ^ 

Product 

u 
> 

< 

—  r»l 

fl  i"1 

OO  fx. 

r— ,  O 

■V  — i 
<xi  in 

rsi  ox 

Raw 

C 
c- 

100.0 
100.0 

100.0 
100.0 

76.1 
79.2 

6.5 
12.5 

21.7 
10.4 

27.2 
50.0 

1 

On  -*r 

in  oo 

OX  rr 

IN  OO 

ox  -ir 

in  oo 

OX  TT 

<N  oo 
OX 

IN  OO 
OX  V 

rnase 

»— •  m 

l*x 

<-l  r*x 

Microbial 
Factor 

u 

0 

iA 

1*1 

® 

u 

3 

u 

0 

o 

IN 
@ 

u 

Si 

Colifonms 

"o 

5.  aureus 

GFAs 

T3 
U 

N 

*| 

§  "33 


CO 

u 


q.  on 


O 

i-  t<- 

u  o 

x>  *- 

S  c 

3  o 

^  CD 

II  Ph 


on 
O 
> 


O 

<u  o 


.—  on 

S  ° 

o 

°  s 

2  O 

o  cs 

r~  > 

O 

ccj  co 

>  § 


o  >  "E 

2  PL,   <  CO 


58 


Figure  1.  APC@35°C  Distributions  -  Plant  RF1 
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Figure  2.  APC@20°C  Distributions  -  Plant  RF1 
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following  packaging  is  indicative  of  the  handling  received.  There  were  no  practical 
differences  between  Phase  I  and  III  results. 

Run  charts  for  the  six  quantitative  microbiology  factors  are  presented  in  Figure  3. 
Centerlines  (CL)  could  not  be  computed  for  any  of  the  factors  because  there  were  insufficient 
positive  results. 

All  93  Listeria  species  swabs  were  negative  in  Phase  I,  and  all  48  were  negative  in 
Phase  III.  This  indicates  effective  sanitation  procedures  were  used  during  both  phases. 

As  discussed  in  the  Methodology  Chapter,  chemistry  testing  was  not  performed  on  any 
of  the  Refrigerated  Foods  plants  because  the  products  were  not  amenable  to  the  chemistry 
factors  selected. 

Refrigerated  Foods  plant  2  (RF  2): 

Table  26  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at  the 
different  sampling  points  for  RF  2.  The  table  relates  the  sampling  points  to  the  generic 
HACCP  Refrigerated  Foods  model  and  demonstrates  the  change  in  microbial  profile  during 
the  process.  As  seen  in  the  table,  the  microbiological  levels  of  incoming  raw  material  are 
low.  The  cooking  process  further  reduced  the  levels  of  microorganisms  —  in  many  cases,  to 
below  the  level  of  detection.  The  levels  of  microorganisms  in  the  final  product  are  slightly 
higher,  reflecting  post-cooking  handling  of  the  product;  however,  the  overall  microbiological 
profile  of  the  final  product  in  both  Phase  I  and  Phase  III  is  indicative  of  good  processing 
procedures.  In  general,  there  was  a  lower  proportion  of  samples  containing  detectable 
microorganisms  in  Phase  III  than  in  Phase  I;  however,  there  was  little  practical  difference 
between  the  microbial  levels  in  final  product  sampled  during  Phase  I  and  final  product 
sampled  during  Phase  III. 

Table  27  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III.  Only  APC  @  35°C  and  APC  @  20°C  showed 
appreciable  percents  positive  in  the  final  product  (see  Pet  values).  The  APC  @35°C  and  APC 
@20°C  distributions  reflect  the  effectiveness  of  the  cooking  process.  More  samples  tested 
negative  in  Phase  III  than  in  Phase  I  (See  Figures  4  and  5). 
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Figure  3.  Run  Charts  for  Microbiology  Data 
Refrigerated  Foods  Plant  1  -  Final  Product 
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Figure  3  (continued).  Run  Charts  for  Microbiology  Data. 
Refrigerated  Foods  Plant  1  -  Final  Product 
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TABLE  26:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  REFRIGERATED  FOODS  PLANT  2 


Process  Step  APC  @  35°C        APC  @  20°C         Coliforms  £-.  coli  S.  aureus  GFAs 

Receiving 

I 

Storage 

I 

Preparation 


Cooking* 
Chilling* 

Assembly* 


Flush  (Gas)* 
I 

Seal  Package 

I 

Inspect  Package* 
I 

Labeling,  etc.* 

I 

Chilling* 


Storage* 


671  1,180  2  1  1  1 

301  218  2  1  11 


|  7  4  1111 

—  3  2  1111 


7  9  1111 

2  2  1111 


Notes: 

1 .  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  All  values  are  reported  on  a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=10)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Figure  4.  APC@35°C  Distributions  -  Plant  RF2 
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Figure  5.  APC@20°C  Distributions  -  Plant  RF2 
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Run  charts  for  the  six  quantitative  microbiology  factors  are  presented  in  Figure  6. 
There  were  sufficient  positive  results  to  compute  a  centerline  (CL)  from  Phase  I  data  for  APC 
@  35°C  and  APC  @  20°C.  There  are  runs  of  points  below  the  centerline  in  Phase  III  for 
both  APC  @  35  (starting  on  days  252  and  294)  and  APC  @  20  (starting  on  days  252  and 
305).  TKrs  indicates  that  a  shift  to  a  lower  mean  count/gm  occurred  in  Phase  III  for  these 
factors. 

All  69  Listeria  species  swabs  were  negative  in  Phase  I.  One  of  41  swabs  was  positive 
in  Phase  III.  The  occasional  presence  of  Listeria  species  in  environmental  samples  is  not 
uncommon,  but  indicates  the  need  to  monitor  cleaning  and  disinfection  procedures  closely. 

In  summary,  RF  2  showed  a  decrease  in  mean  count/gm  for  both  APC  @35°C  and 
APC  @  20°C.  The  microbiological  profile  of  product  produced  during  both  phases  was 
indicative  of  good  manufacturing  procedures  including  effective  process  control. 

Refrigerated  Foods  plant  3  (RF  3): 

Table  28  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at  the 
different  sampling  points  at  RF  3,  during  Phases  I  and  III.  The  table  relates  the  sampling 
points  to  the  generic  HACCP  Refrigerated  Foods  model  and  demonstrates  the  change  in 
microbial  profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  of 
incoming  raw  material  were  low.  The  cooking  process  reduced  the  levels  of  microorganisms 
even  further  —  in  many  cases,  to  the  level  of  detection  or  below.  The  levels  of 
microorganisms  in  the  final  product  are  slightly  higher,  reflecting  post-cooking  handling  of 
the  product  or  combination  with  uncooked  ingredients;  however,  the  overall  microbiological 
profile  of  the  final  product  in  both  Phase  I  and  Phase  III  is  indicative  of  good  processing 
procedures. 

Table  29  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III.  There  were  a  greater  number  of  samples  for  final 
product  because  duplicate  final  product  samples  were  collected  each  sampling  day  in  this 
plant.  Only  APC  @  35°C  and  APC  @  20°C  showed  appreciable  percents  positive  in  final 
product  (see  Pet  values).  Figures  7  and  8  display  RF  3's  APC  @  35°  and  20°  C  results.  The 
results  show  the  effectiveness  of  the  cooking  process  in  reducing  overall  microbial  levels. 
The  effect  of  post-cooking  handling  is  shown. 
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Figure  6.  Run  Charts  for  Microbiology  Data 
Refrigerated  Foods  Plant  2  -  Final  Product 
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Figure  6  (continued).  Run  Charts  for  Microbiology  Data. 
Refrigerated  Foods  Plant  2  -  Final  Product 
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TABLE  28:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  REFRIGERATED  FOODS  PLANT  3 


Process  Step  APC  @  35°C  APC  @  20°C  Colifonms  £.  coli  S.  aureus  GFAs 
 r-  ~~" ^ ~ ~ ~ ~— ^ ~~~ ~ ~ "~ 


Receiving 

I 

Storage 

I 

Preparation 

I 
I 

Cooking* 

I 

Chilling* 

I 
I 

Assembly* 

I 

Flush  (Gas)* 

I 

Seal  Package 

I 

Inspect  Package* 
I 

Labeling,  etc.* 

I 

Chilling* 

I 

I 

Storage* 


38,100  215,000  239  20  2  8 

85,500  842,000  145  59  1  8 


6  5  l  —if"         "ia  i 

4  3  1111 


31  19  1  1  1  1 

83  51  1  1  1  1 


Notes: 

1 .  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  All  values  are  reported  on  a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=10)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Figure  7.  APC@35°C  Distributions  -  Plant  RF3 
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Figure  8.  APC@20°C  Distributions  -  Plant  RF3 
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Run  charts  for  the  six  quantitative  microbiology  factors  are  presented  in  Figure  9.  There  were 
sufficient  positive  results  to  compute  a  centerline  (CL)  for  APC  @  35°C  and  APC  @  20°C. 
There  is  a  run  of  points  above  the  centerline  in  Phase  III,  starting  at  day  436,  for  APC  @ 
35°C.  This  indicates  that  a  shift  to  a  higher  mean  count/gm  occurred  in  Phase  III  for  this 
factor.  The  pattern  for  APC  @  20°C  is  similar,  although  these  is  no  run  of  eight  consecutive 
points  above  the  CL.  Levels  for  all  other  factors  in  Phase  III  were  equal  or  less  than  those 
in  Phase  I. 

All  57  Listeria  species  swabs  were  negative  in  Phase  I,  and  all  23  were  negative  in 
Phase  III. 

In  summary,  RF  3  showed  a  shift  to  higher  mean  count/gm  for  APC  @  35°C.  The 
microbiological  profile  of  product  produced  during  both  phases  is  indicative  of  good 
manufacturing  procedures. 

Cooked  Sausage  Plants: 

The  quantitative  data  for  the  three  Cooked  Sausage  plants  included  seven  microbiology 
factors  and  two  chemistry  factors.  The  seven  microbiology  factors  are  listed  in  Chapter  2: 

Samples. for  the  first  six  factors  were  collected  at  three  different  control  points  in 
processing:  (1)  after  stuffing;  (2)  after  chilling  and  storage;  and  (3)  from  final  packaged 
product.  Results  for  these  factors  were  reported  in  counts  per  gram.  Swabs  for  Listeria- 
species  tests  were  taken  from  four  different  finished  product  contact  surfaces  on  an  alternating 
basis.  Results  for  Listeria  were  reported  only  as  positive  or  negative  (not  quantified). 

The  sampling  sites  were  purposely  selected  to  bracket  (i.e.,  before  and  after)  critical 
control  points  in  the  production  of  cooked  sausages.  These  critical  points  are  those  related 
to  cooking,  chilling,  and  post-cooking  handling  of  the  product.  The  cooking  process  is  the 
only  critical  control  point  in  the  production  process  for  this  product  where  appreciable 
reduction  of  microbial  levels  can  be  expected.  The  sampling  sites  are  depicted  in  Tables  30, 
32  and  34  in  relation  to  both  the  critical  control  points  and  other  steps  in  the  production 
process.  The  mean  microbiological  levels  for  each  of  the  six  factors  at  each  of  the  sampling 
sites  are  shown  in  the  same  tables.  The  values  in  the  tables  for  points  prior  to  cooking  show 
the  microbiological  condition  of  the  raw  material.  The  values  immediately  after  cooking  and 
chilling  indicate  the  effectiveness  of  the  cooking  process  in  reducing  microbial  levels  and  the 
effectiveness  of  the  chilling  process  in  retarding  multiplication  of  any  survivors.  The  final 
values  in  the  tables  indicate  the  degree  of  sanitation  practiced  during  the  steps  of  the  process 
leading  to  in  the  final  packaged  product.  In  these  tables,  as  indicated,  a  value  of  one  was 
assigned  to  samples  which  were  negative  to  facilitate  the  calculation  of  the  geometric  mean. 
These  tables  show  the  relative  levels  for  each  microbiological  factor  at  the  various  stages  in 
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Figure  9.  Run  Charts  for  Microbiology  Data 
Refrigerated  Foods  Plant  3  -  Final  Product 
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Figure  9  (continued).  Run  Charts  for  Microbiology  Data. 
Refrigerated  Foods  Plant  3  -  Final  Product 
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the  production  process  and  present  what  is  known  as  the  "microbiological  profile"  of  the 
production  process.  Tables  31,33,35  present  the  percent  of  samples  testing  positive  for  each 
of  the  microbiological  factors  and  the  log  average  level  for  each. 

The  two  chemistry  factors  were  nitrites  and  salt  (brine).  Samples  of  final  product  were  tested 
for  nitrites  and  salt  (brine).  In  addition,  samples  of  curing  mixtures  were  tested  for  nitrites. 

The  plan  to  collect  sample  sets  on  30  to  50  different  days  during  both  the  baseline  and 
operational  phases  (Phases  I  and  III)  was  fulfilled  for  all  three  Cooked  Sausage  plants.  The 
results  were  analyzed  to  check  each  plant's  ability  to  control  its  process  and  to  compare 
characteristics  of  product  produced  in  Phases  I  and  III.  Results  are  discussed  separately  for 
each  plant.  Results  for  microbiology  and  chemistry  factors  from  Phase  I  were  shared  with 
the  plants  at  the  beginning  of  Phase  II,  and  results  from  Phase  II  were  shared  as  they  became 
available. 

Cooked  Sausage  plant  1  (CS  1) 

Table  30  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at  the 
different  sampling  points  in  CS  1  during  Phases  I  and  III.  It  also  relates  the  sampling  points 
to  the  generic  HACCP  Cooked  Sausage  model  and  demonstrates  the  change  in  microbial 
profile  during  the  process.  The  microbiological  levels  of  incoming  raw  material  were  low. 
The  cooking  process  further  reduced  the  level  of  microorganisms  --  in  many  cases,  to  below 
the  level  of  detection.  The  level  of  microorganisms  in  the  final  product  was  similar  to  that 
just  after  cooking  and  chilling,  indicating  good  handling  procedures  during  peeling  and 
packaging.  The  overall  microbiological  profile  of  the  final  product  in  both  Phases  I  and  III 
was  indicative  of  good  processing  procedures.  There  was  little  practical  difference  between 
the  microbial  profiles  of  product  sampled  during  Phase  I  versus  Phase  III. 

Table  31  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III.  There  are  greater  numbers  of  samples  for  final 
product  because  duplicate  final  product  samples  were  collected  each  sampling  day  in  this 
plant.  Only  APC  @  35°C  and  APC  @  20°C  showed  appreciable  percent  positive  in  final 
product  (see  Pet  values).  Figures  10  and  11  display  the  APC  @  35°C  and  APC  @  20°C 
distribution  and  demonstrate  the  effectiveness  of  the  cooking  process.  Little  effect  of  post- 
cooking  handling  was  evident.  Phase  I  and  Phase  III  results  are  similar. 

Run  charts  for  the  six  quantitative  microbiology  factors  in  final  product  are  presented 
in  Figure  12.  There  were  sufficient  positive  results  to  compute  a  centerline  (CL)  from  Phase 
I  data  for  APC  @  35°C  and  APC  @  20°C.  For  APC  @  35°C  there  were  runs  of  points 
below  the  centerline  in  Phase  I  (starting  on  day  73)  and  in  Phase  III  (starting  on  day  343). 
This  indicates  that  shifts  to  lower  mean  count/gram  occurred  for  this  factor. 
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TABLE  30:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  COOKED  SAUSAGE  PLANT  1 


Process  Step  APC  @  35°C       APC  @  20°C        Coliforms  E.  coli  S.  aureus  GFAs 

Receiving 

I 

Storage 

I 

Preparation 

.  I 

Formulation* 

I 

Chop/Blend/Emulsify* 
I 

Stuffing 

|  4,180  11,600  3  2  3  4 

—  4,140  13,800  3  1  2  2 
I 

Cook/Smoke* 
I 

Showering 

I 

Chill/Storage* 

|  6  4  1111 

—  7  8  1111 


Peeling* 

I 

Packaging* 

I 
I 

Storage* 

I 

Shipping/Distribution 


7  8  1111 

6  8  1111 


Notes: 

1 .  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  All  values  are  reported  on  a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=10)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Figure  10.  APC@35°C  Distributions  -  Plant  CS1 
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Figure  11.  APC@20°C  Distributions  -  Plant  CS1 
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Figure  12.   Run  Charts  for  Microbiology  Data 
Cooked  Sausage  Plant  1  -  Final  Product 
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Figure  12  (continued).  Run  Charts  for  Microbiology  Data- 
Cooked  Sausage  Plant  1  -  Final  Product 
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All  56  Listeria  species  swabs  were  negative  in  Phase  I.  and  one  of  42  was  positive  in 
Phase  III.  The  occasional  presence  of  Listeria  species  in  envn'onmentaJ  samples  is  not 
uncommon,  but  indicates  the  need  to  monitor  cleaning  and  disinfection  procedures  closely. 


Table  32  displays  the  chemistry  testing  results  of  finished  product  in  Phase  I  and  III. 
TABLE  32:  NITRITE  AND  BRINE  CHEMISTRY  TESTING  RESULTS  FOR  CS  1 


Analyie 

Phase 

N 

Mean 

Minimum 

Maximum 

Nitrite 

t 

58 

33.4 

11.0 

52.0 

3 

33 

42.6 

33.0 

52.C 

Brine 

I 

5S 

3.9 

J  .J 

+  .7 

3 

33 

3.9 

3.5 

4.6 

The  mean  nitrite  values  (Table  32)  for  Phases  I  and  III  were  significantly  lower  than 
maximum  allowable  amount  of  156  parts  per  million  (ppm)  in  the  formulation  of  the  product, 
also  the  maximum  values  observed  for  both  phases  (52  ppm)  were  significantly  lower  than 
the  maximum  allowable  value  of  156  ppm.  It  can  be  seen  that  there  is  a  large  variation  in 
the  extent  of  nitrite  depletion  between  individual  products.  This  is  since  sodium  nitrite  is 
relatively  unstable  and  small  variations  in  storage  tune,  storage  temperature,  and  matrix 
composition  can  have  large  effects  upon  the  depletion  rate  of  this  analyte.  The  percentage 
of  brine  seen  in  the  above  table  is  within  the  normal  range  for  cooked  sausages  in  Phases  I 
and  III. 

In  summary,  the  HACCP  microbiological  assessment  in  Cooked  Sausage  Plant  CS 
1  showed  a  decrease  in  mean  count/gram  for  APC  @  35  °C  and  APC  @  20°C,  however,  the 
overall  microbiological  profile  of  product  produced  during  both  phases  was  indicative  of 
good  manufacturing  procedures. 

Cooked  Sausage  plant  2  (CS  2) 

Table  33  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at  the 
different  sampling  points  in  CS  2  during  Phases  I  and  III.  It  also  relates  the  sampling  points 
to  the  generic  HACCP  Cooked  Sausage  model  and  demonstrates  the  change  in  microbial 
profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  of  incoming  raw 
material  were  low.  The  cooking  process  further  reduced  the  levels  of  microorganisms.  The 
levels  of  microorganisms  in  the  final  product  were  similar  to  those  just  after  cooking  and 
chilling,  indicating  good  handling  procedures  during  peeling  and  packaging.  The  overall 
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TABLE  33:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  COOKED  SAUSAGE  PLANT  2 


Process  Step  APC  @  35°C       APC  @  20°C        Coliforms  £.  coti  5.  aureus  GFAs 

Receiving 

I 

Storage 

I 

Preparation 

I 

Formulation* 
I 

Chop/Blend/Emulsify* 

I  - 

Stuffing 

|  114,000  337,000  279  5  2  4 

—  52,100  184,000  50  3  4  2 

I 

Cook/Smoke* 

I 

Showering 

I 

Chill/Storage* 


! 

Peeling* 

I 

Packaging* 


Storage* 

I 

Shipping/Distribution 


37  27  1  1  1  1 

30  35  1  1  1  1 


35  46  1111 

24  23  1  1  1  1 


Notes: 

1 .  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  All  values  are  reported  on  a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=10)  were  assigned  a  value 
of  1  in  computing  the  geometric  mean. 
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microbiological  profile  of  the  final  product  in  both  Phases  I  and  III  was  indicative  of  good 
processing  procedures,  and  there  was  little  practical  difference  between  them. 

Table  34  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III.  There  were  greater  numbers  of  samples  for  final 
product  because  duplicate  samples  were  collected  each  sampling  day  in  this  plant.  Only  APC 
@  35°C  and  APC  @  20°C  showed  appreciable  percent  positives  in  final  product  (see  Pet 
values).  Figures  13  and  14  display  the  APC  @  35°C  and  20°C  distributions.  Little  increase 
in  microbial  levels  following  cooking  was  noted  in  Phase  I  and  Phase  III. 

Run  charts  for  the  six  quantitative  microbiology  factors  in  final  product  are  presented 
in  Figure  15.  There  were  sufficient  positive  results  to  compute  a  centerline  (CL)  from  Phase 
I  data  for  APC  @  35°C  and  APC  @  20°C,  if  appropriate.  There  are  runs  of  points  below  the 
centerline  in  Phases  I  and  III  for  APC  @  35°C  (starting  on  days  10  and  486)  and  for  APC  @ 
20°C  (starting  on  days  1 5  and  486);  there  are  runs  above  the  centerline  in  Phase  I  for  both 
factors  (starting  on  day  121).  This  variation  indicates  an  unstable  process. 

Results  for  Listeria  species  swabs  were  as  follows:  2  of  51  positive  in  Phase  I,  and  2 
of  45  positive  in  Phase  III.  The  occasional  presence  of  Listeria  species  in  environmental 
samples  is  not  uncommon,  but  indicates  the  need  to  monitor  cleaning  and  disinfection 
procedures  closely. 

The  chemistry  testing  results  of  finished  product  in  Phases  I  and  III  are  displayed  in 
Table  35. 

The  mean  nitrite  values  for  CS  2  were  nearly  the  same  in  Phases  I  and  III  (40.6  ppm 
and  40.8  ppm,  respectively)  and  significantly  lower  than  the  maximum  allowable  level  of  156 
ppm  in  the  formulation  of  the  product.  The  maximum  values  of  65  ppm  and  52  ppm  for 
Phases  I  and  III,  respectively,  are  also  well  below  the  maximum  allowable  limit.  As 
explained  in  the  discussions  of  CS  1 ,  this  wide  variation  is  due  to  the  relative  instability  of 
sodium  nitrite.  The  Phases  I  and  III  percentages  for  brine  show  that  CS  2  is  within  the  normal 
range  for  cooked  sausage  product. 

In  summary,  the  HACCP  microbiological  assessment  in  Cooked  Sausage  Plant  CS  2 
showed  process  instability  for  APC  @  35°C  and  APC  @  20°C;  however,  the  overall 
microbiological  profile  of  product  produced  during  both  phases  was  indicative  of  good 
manufacturing  procedures. 
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Figure  13.  APC@35°C  Distributions  -  Plant  CS2 
Phase  1  Phase  3 


Percent 


60 


60 


40 


20 


Raw  Product  Percent 
100 


Raw  Product 


too 


*"! — i  i  i  i — i — i — r  i  i — ill  ill 

0.00      100      2.00      3.00      4.00      5.00      6.00      7.00        0  00       1.00      2.00      3  00      4.00      5  00      6  00  7.00 


Log,0(Count  +  1) 


Log10(Count+1) 


Percent 


100 


80 


60 


20  -  • 


Cooked  Product  Percent 
100 1  


Cooked  Product 


1 1 1 1 1 1 1 1 


ao 


60  - 


40 


20  - 


0^ 


i  i  i   i  i — i  1 1"  i   1   i  i — i   i  i  i — i — r 
0.00       1.00       2.00       3.00       4  00       5.00       6.00       7.00         0.00       1.00       2.00       3.00       4.00       5.00       6.00       7  00 


Log,0(Count+l) 


Log,0(Count+1) 


Percent 


LJL. 



Final  Product  Percent 
100 1  


Final  Product 


80 


60 


40  - 


20 


0.00       1.00       2.00       3.00       4.00       5.00       6.00       7.00         0.00       1.00       2.00       3.00       4.00       5.00       6.00       7  00 
Log10(Count  +  1)  Logl0(Count+1) 


Figure  14.  APC@20°C  Distributions  -  Plant  CS2 
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Figure  15.  Run  Charts  for  Microbiology  Data 
Cooked  Sausage  Plant  2  -  Final  Product 
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Figure  15  (continued).  Run  Charts  for  Microbiology  Data. 
Cooked  Sausage  Plant  2  -  Final  Product 
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TABLE  35:  NITRITE  AND  BRINE  CHEMISTRY  TESTING  RESULTS  FOR  CS  2 


Analyte 

Phase 

N 

Mean 

Minimum 

Maximum 

Nitrite 

1 

75 

40.6 

16.0 

65.0 

3 

14 

40.8 

27.0 

52.0 

Brine 

1 

75 

4.2 

3.7 

4.8 

3 

H 

4.0 

3.9 

4.2 

Cooked  Sausage  plant  3  (CS  3): 

Table  36  shows  geometric  means  for  the  six  quantitative  microbiology  factors  for 
CS  3  at  the  different  sampling  points  during  Phases  I  and  III.  It  also  relates  the  sampling 
points  to  the  generic  HACCP  Cooked  Sausage  model  and  demonstrates  the  change  in 
microbial  profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  of 
incoming  raw  material  were  low.  The  cooking  process  further  reduced  the  levels  of 
microorganisms.  The  levels  of  microorganisms  in  the  final  product  were  similar  to  those 
just  after  cooking  and  chilling,  indicating  good  handling  procedures  during  peeling  and 
packaging.  The  overall  microbiological  profile  of  the  final  product  in  both  Phases  I  and 
III  was  indicative  of  good  processing  procedures  and  there  was  little  practical  difference 
between  them. 

Table  37  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III.  There  were  greater  numbers  of  samples  for  final 
product  because  duplicate  samples  were  collected  each  sampling  day  in  this  plant.  Only 
APC  @  35°C  and  APC  @  20°C  showed  appreciable  percent  positives  in  final  product  (see 
Pet  values).  Figures  16  and  17  display  the  APC  @  35°C  and  20°C  distributions.  Little 
increase  in  microbial  levels  following  cooking  was  noted  in  Phase  I  and  Phase  III. 

Run  charts  for  the  six  quantitative  microbiology  factors  in  final  product  are 
presented  in  Figure  18.  There  were  sufficient  positive  results  to  compute  a  centerline 
(CL)  from  Phase  I  data  for  APC  @  35°C  and  APC  @  20°C.  There  are  runs  of  points 
below  the  centerline  in  Phase  I  and  III  for  APC  @  35°C  (starting  on  days  86  and  227) 
and  for  APC  @  20°C  (starting  on  days  86  and  248);  there  are  runs  above  the  centerline  in 
Phase  I  for  both  factors  (starting  on  day  43).  This  variation  indicates  an  unstable  process. 

Results  for  Listeria  species  swabs  were  as  follows:  0  of  69  positive  in  Phase  I, 
and  1  of  47  positive  in  Phase  III.  The  occasional  presence  of  Listeria  species  in 
environmental  samples  is  not  uncommon,  but  indicates  the  need  to  monitor  cleaning  and 
disinfection  procedures  closely. 
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TABLE  36:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  COOKED  SAUSAGE  PLANT  3 

Process  Step  APC  @  35°C       APC  @  20°C        Coliforms  E.  coli  S.  aureus  GFAs 

Receiving 

I 

Storage 

I 

Preparation 

I 

Formulation* 
I 

Chop/Blend/Emulsify* 


Stuffing 

I 
I 

Cook/Smoke* 
I 

Showering 

I 

Chill/Storage* 


I 

Peeling* 


Storage* 

I 

Shipping/Distribution 


9,630  18,100  9  2  2  5 

7,560  33,500  6  2  5  3 


30  19  1  1  1  1 

14  10  1  1  11 


Packaging* 

|  27  22    '  1  1  11 


12  7  1111 


Notes: 

1.  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  All  values  are  reported  on  a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=10)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Figure  16.  APC@35°C  Distributions  -  Plant  CS3 
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Figure  17.  APC@20°C  Distributions  -  Plant  CS3 
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Figure  18.  Run  Charts  for  Microbiology  Data 
Cooked  Sausage  Plant  3  -  Final  Product 
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Figure  18  (continued).  Run  Charts  for  Microbiology  Data. 
Cooked  Sausage  Plant  3  -  Final  Product 


Mean  Log10(Count+1) 


coli 


Phase  1 


Phase  2 


Phase  3 


2  — 


Mean  Log1Q(Count+  1 ) 


T 


S.  aureus 


1  — 


I 


A 


SO  1 0O         1  50         200  250 

Day  Number 


300         350  400 


Mean  Log10(Count+  1 ) 


GFAs 


4  — 


3  — 


1  — 


50  1 0O         1  50         200  250 

Day  Number 


300         350  400 


The  chemistry  testing  results  of  finished  product  for  nitrite  and  brine  in  Phases  1 
and  III  are  displayed  in  Table  38. 


TABLE  38:  NITRITE  AND  BRINE  CHEMISTRY  TESTING  RESULTS  FOR  CS  3 


Analyte 

Phase 

N 

Mean 

Minimum 

Maximum 

Nitnte 

1 

74 

48.5 

16.0 

83.0 

3 

48 

33.9 

6.0 

57.0 

Bnne 

1 

74 

3.4 

3.0 

4.1 

3 

48 

3.4 

3.2 

3.7 

As  seen  in  Table  38,  CS  3's  mean  nitrite  values  for  both  Phase  I  and  III  are  well 
below  the  maximum  allowable  level  of  156  ppm  and  in  the  formulation  of  the  product. 
The  nitrite  maximum  value  for  Phase  III  (57.0  ppm)  was  lower  than  the  Phase  I  value  of 
83.0  ppm.  As  previously  discussed  for  CS  1  and  CS  2,  these  ranges  are  due  to  the 
instability  of  sodium  nitrite.  The  brine  percentages  for  both  phases  are  within  the  normal 
range  for  cooked  sausage  product. 

In  summary,  the  HACCP  microbiological  assessment  in  Cooked  Sausage  Plant  CS 
3  showed  process  instability  for  APC  @  35°C  and  APC  @  20°C;  however,  the  overall 
microbiological  profile  of  product  produced  during  both  phases  was  indicative  of  good 
manufacturing  procedures. 

Poultry  Slaughter  Plants: 

The  quantitative  data  for  Poultry  Slaughter  plants  included  the  six  microbiology 
factors  discussed  in  Chapter  II.  For  all  six  factors,  samples  were  collected  at  four 
different  control  points  in  the  process:  (1)  before  scalding;  (2)  after  hock  cutting;  (3)  after 
final  wash;  and  (4)  after  carcass  chill.  Results  for  the  first  five  factors  were  reported  in 
counts  per  gram;  results  for  Salmonella  were  reported  as  counts  per  bird. 

The  sampling  sites  were  purposely  selected  to  bracket  (before  and  after)  certain 
critical  control  points  during  the  processing  of  broiler  chickens.  In  contrast  to  the 
production  of  cooked,  refrigerated  foods  and  cooked  sausages,  no  single  critical  control 
point  during  this  production  process  can  be  relied  on  to  assure  the  destruction  of  the 
microorganisms  generally  associated  with  live  broilers.  The  sampling  sites  are  depicted  in 
Tables  39,  41  and  43  in  relation  to  both  the  critical  control  points  and  other  steps  in  the 
production  process.  The  mean  microbiological  levels  for  each  of  the  factors  at  each  of  the 
sampling  sites  are  shown  in  the  same  tables.  The  values  in  the  tables  prior  to  scalding 
show  the  microbiological  condition  of  the  incoming  birds  at  the  earliest  possible  time  in 
the  production  process.  The  values  following  the  tables  show  the  effect  of  the  various 
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other  process  steps  on  the  microbiological  condition.  In  these  tables,  as  indicated  a  value 
of  one  was  assigned  to  samples  which  were  negative  to  facilitate  the  calculation  of  the 
geometric  mean.  These  tables  show  the  relative  levels  of  each  microbiological  factor  at 
the  various  stages  in  the  production  process  and  present  what  is  known  as  the 
"microbiological  profile"  of  the  production  process.    Tables  40,  42,  and  44  present  the 
percent  of  samples  testing  positive  for  each  of  the  microbiological  factors  and  the  log 
average  level  for  each. 

One  chemistry  factor,  chlorine  in  processing  water  was  included.  Chlorine  in 
chiller  water  was  tested  as  a  possible  explanatory  variable  in  case  of  process  control 
failures.  Values  were  in  units  of  10  ppm  (i.e.,  0  ppm,  10  ppm,  20  ppm,  etc.).  The 
maximum  permitted  level  is  50  ppm. 

The  plan  to  collect  sample  sets  on  30  to  50  different  days  during  both  the  baseline 
and  operational  phases  (Phases  I  and  III)  was  fulfilled  for  all  three  poultry  slaughter 
plants.  The  results  were  analyzed,  as  outlined  in  the  evaluation  plan,  to  check  each  plant's 
ability  to  control  its  process  and  to  compare  characteristics  of  product  produced  in  Phases 
I  and  III.  Results  are  discussed  separately  for  each  plant.  Results  for  microbiology  and 
chemistry  factors  from  Phase  I  were  shared  with  the  plants  at  the  beginning  of  Phase  II, 
and  results  from  Phase  II  were  shared  as  they  became  available. 

Poultry  Slaughter  plant  1  (PS  1): 

Table  39  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at 
the  different  sampling  points  for  PS  1  during  Phases  I  and  III.  It  also  relates  the  sampling 
points  to  the  generic  HACCP  Poultry  Slaughter  model  and  demonstrates  the  change  in 
microbial  profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  were 
high  for  incoming  birds  but  each  successive  processing  step  reduced  the  microbial 
population  on  the  birds. 
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TABLE  39:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  POULTRY  SLAUGHTER  PLANT  1 


Process  Step  APC  @  35CC  Coliforms  E.  colt  S.  aureus  GFAs  Salmonella 


Receiving* 

Hanging 

Stunning 

Killing 

Bleeding 

2,180,000 

1,270 

632 

5 

234 

1 

3,890,000 

1,330. 

624 

2 

178 

2 

Scalding* 

Picking 

Singeing 

Remove  Head 

Washing 

Hock  cutter 

9.060 

234 

148 

10 

59 

1 

1   

6,560 

130 

69 

2 

7 

1 

Trans  f/rehang 

Oil  Sac  ait 

Vent/etc* 

Presenting 

Inspection 

Helper 

Liver/Heart 

Gut  cutter 

Gizzard 

Cropping 

Break  Neck 

Remove  Lung 

Trimmer 

House  Check 

Final  Wash* 

1,950 

17 

10 

9 

10 

1 

2,010 

23 

13 

24 

3 

1 

Neck  cutter 

Carcass  chill* 

185 

3 

2 

1 

2 

1 

347 

7 

4 

4 

2 

1 

Sort/etc. 

Package/etc. 

Storage/Dist* 

Notes: 

1 .  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  Salmonella  was  reported  on  a  per  bird  basis;  other  values  were  reported  on 
a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=1)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Table  40  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  four 
sampling  points  during  Phases  I  and  III. 

Figures  19-24  show  the  distribution  for  APC  @  35°C,  coliforms,  E.  coli,  S. 
aureus,  GFA,  and  Salmonella.  The  results  generally  show  decreases  in  all  factors  as  the 
product  moves  through  successive  stages  of  processing.  Phase  I  and  Phase  III  results  are 
similar. 

Run  charts  for  the  six  quantitative  microbiology  factors  are  presented  in  Figure  25. 
There  were  sufficient  positive  results  to  compute  a  centerline  (CL)  from  Phase  I  data  for 
APC  @  35°C  ,  coliforms,  and  GFAs.    For  APC  @  35°C  and  coliforms,  there  are  runs  of 
points  above  the  centerline  in  Phase  III  (starting  on  days  288  and  341  for  APC  @  35°C 
and  on  days  285  and  358  for  coliforms).  This  variation  indicates  that  a  shift  to  higher 
process  mean  count/gram  occurred  in  Phase  III  for  these  factors.  Figure  24  and  Table  40 
show  that  Salmonella  results  for  Phase  III  tended  to  he  higher  than  those  for  Phase  I. 
This  difference  was  to  be  expected,  since  the  Salmonella  counts  for  the  incoming  birds  in 
Phase  III  were  higher  than  in  Phase  I. 

Percent  positive  Salmonella  was  proposed  to  be  used  for  process  control 
verification;  therefore  percent  positives  for  Phases  I  and  III  were  compared.  The 
difference,  3.5  percent  vs  15.4  percent,  was  found  to  be  statistically  significant  (P<.05). 

The  median  chlorine  levels  were  30  ppm  in  Phase  I  and  20  ppm  in  Phase  III.  The 
chlorine  level  was  lowered  around  day  250.  This  change  may  account  in  part  for  higher 
microbial  levels  in  Phase  III. 

In  summary,  the  HACCP  microbiological  assessment  in  Poultry  Slaughter  Plant  PS 
1  showed  a  shift  to  higher  microbial  levels  and  an  increase  in  Salmonella  in  Phase  III. 
However,  the  microbiological  profile  of  the  product  produced  during  both  phases  was 
indicative  of  good  manufacturing  procedures. 

Poultry  Slaughter  plant  2  (PS  2): 

Table  41  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at 
the  different  sampling  points  in  PS  2  during  Phases  I  and  III.  It  also  relates  the  sampling 
points  to  the  generic  HACCP  Poultry  Slaughter  model  and  demonstrates  the  change  in 
microbial  profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  were 
high  for  incoming  birds  but,  each  successive  step  reduced  the  microbial  population  on  the 
birds. 
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Figure  19.  APC@35°C  Distributions  -  Plant  PS1 
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Figure  20.  Coliform  Distributions  -  Plant  PS1 
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Figure  21.  E.  coli  Distributions  -  Plant  PS1 
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Figure  22.  S.  aureus  Distributions  -  Plant  PS1 
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Figure  23.  GFA  Distributions  -  Plant  PS1 
Phase  1  Phase  3 
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Figure  24.  Salmonella  Distributions  -  Plant  PS1 
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Figure  25.  Run  Charts  for  Microbiology  Data 
Poultry  Slaughter  Plant  1  -  After  Chill  Tank 
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Figure  25  (continued).  Run  Charts  for  Microbiology  Data 
Poultry  Slaughter  Plant  1  -  After  Chill  Tank 
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TABLE  41:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  POULTRY  SLAUGHTER  PLANT  2 


Process  Step  APC  @  35°C  Coliforms  £.  co/i  S.  aureus  GFAs  Salmonella 


Receiving* 

Hanging  ' 

Stunning 

k  line 

Bleeding 

1  190  000 

2,770 

1,430 

g 

588 

IS 

1  — 
I 

4,790,000 

3,590-  . 

2,180 

22 
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2 

Sra  In  i  np* 

Picking 

Singeing 

Remove  Held 
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H If  *~iirT/»r 

|   

R  980 

1  JO 

so 

JO 

If! 

JO 

1 

T  raiis  {/  re  lung 

1 1  700 

246 
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22 

4 

1 

Oil  Sac  cut 

Vent/ etc.  * 

Presenting 
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Liver/Heart 
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Break  Neck 
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Trimmer 

House  Check 

Final  Wash* 
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5 

3 

1 

5 
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18 
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2 

1 

Carcass  chill* 
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9 

3 

1 

4 

2 

Sort/ etc. 

1,180 

18 

7 

3 

2 

1 

Package/ etc. 

Siorage/Dist* 

Notes: 

1.  *  denotes  microbiological  Critical  Control  Point  (CCP) 

2.  Salmonella  was  reported  on  a  per  bird  basis;  other  values  were  reported  on 
a  per  gram  basis. 

3.  Results  below  the  Minimum  Detectable  Level  (MDL=1)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Table  42  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III. 

Figures  26  through  31  show  the  distribution  for  APC  @  35°C,  coliforms,  E.  coli, 
S.  aureus^GFA,  and  Salmonella.  The  results  generally  show  decreasing  levels  for  all 
factors  as  the  product  moves  through  processing.  Phase  I  and  Phase  III  results  are  similar. 

Run  charts  for  the  six  quantitative  microbiology  factors  are  presented  in  Figure  32. 
There  were  sufficient  positive  results  to  compute  a  centerline  (CL)  from  Phase  I  data  for 
APC  @  35°C  ,  coliforms,  E.  coli,  and  GFAs. 

Percent  positive  Salmonella  was  proposed  to  be  used  for  process  control 
verification;  therefore,  percent  positives  for  Phases  I  and  III  were  compared.  The 
difference,  30.2  percent  vs  9.8  percent,  was  found  to  be  statistically  significant  (P<.05). 
Figure  3 1  and  Table  42  show  that  Salmonella  results  for  Phase  III  tended  to  be  lower  than 
those  for  Phase  I.  This  difference  was  to  be  expected,  since  the  Salmonella  results  for  the 
incoming  birds  in  Phase  III  were  lower  than  those  in  Phase  I.  The  median  chlorine  levels 
were  10  ppm  in  both  Phase  I  and  Phase  III  and  has  no  explanatory  value. 

In  summary,  the  HACCP  microbiological  assessment  in  Poultry  Slaughter  Plant  PS 
2  showed  a  reduction  in  Salmonella.  The  microbiological  profile  of  the  product  produced 
during  both  phases  was  indicative  of  good  manufacturing  procedures. 

Poultry  Slaughter  plant  3  (PS  3): 

Table  43  shows  geometric  means  for  the  six  quantitative  microbiology  factors  at 
the  different  sampling  points  in  PS  3  during  Phases  I  and  III.  It  also  relates  the  sampling 
points  to  the  generic  HACCP  Poultry  Slaughter  model  and  demonstrates  the  change  in 
microbial  profile  during  the  process.  As  seen  in  the  table,  the  microbiological  levels  were 
high  for  incoming  birds,  but  each  successive  step  reduced  the  microbial  population  on  the 
birds. 

Table  44  gives  more  detailed  summary  statistics  for  the  same  factors  at  the  three 
sampling  points  during  Phases  I  and  III. 

Figures  33  through  38  show  the  distribution  for  APC  @  35°C,  coliforms,  E.  coli, 
S.  aureus,  GFA,  and  Salmonella.  The  results  generally  show  decreasing  levels  for  all 
factors  as  the  product  moves  through  processing.  Phase  I  and  Phase  III  results  are  similar. 
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Figure  26.  APC@35°C  Distributions  -  Plant  PS2 
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Figure  28.  E.  coli  Distributions  -  Plant  PS2 
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Figure  29.  S.  aureus  Distributions  -  Plant  PS2 
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Figure  30.  GFA  Distributions  -  Plant  PS2 
Phase  1  Phase  3 
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Figure  31.  Salmonella  Distributions  -  Plant  PS2 
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Figure  32.  Run  Charts  for  Microbiology  Data 
Poultry  Slaughter  Plant  2  -  After  Chill  Tank 
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Figure  32  (continued).  Run  Charts  for  Microbiology  Data 
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TABLE  43:  MICROBIAL  PROFILE  AT  DIFFERENT  PROCESS  STEPS 
GEOMETRIC  MEAN  -  PHASE  I/PHASE  HI  POULTRY  SLAUGHTER  PLANT  3 
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3.  Results  below  the  Minimum  Detectable  Level  (MDL=1)  were  assigned  a 
value  of  1  in  computing  the  geometric  mean. 
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Figure  33.  APC@35°C  Distributions  -  Plant  PS3 


Phase  1 


Phase  3 


Before  So* Id 


Percent 


Bafore  Scald 


100 


20  - 


ll7rrO" 


DL 


60 


60 


20 


JO 


O.OO    1.00    2.00    3.00    4  00    5.00    6.00    7.00    8.00  9.00 
Log,0(Count  +  l) 


0.00     1.00    2.00    3.00    4.00    5.00    6.00    7.00    6.00  9.00 
Log,0(Count  +  1) 


Percent 


Attar  Hook  Cutting 


Percent 


Altar  Hook  Cutting 


60 


-jfllTfVn 


80 


60 


0.00     1.00    2.00    3  00    4  00    5.00    6.00    7.00    8.00    9  00 
Log,0(Count  +  1) 

Percent  Attar  Final  Waeh 


0  00     1.00    2.00    3.00    4.00    5.00    6.00    7.00    8.00  9.00 
log10(Courrt  +  l) 


Percent 


Attar  Final  Waah 


60  - 


40  - 


20  - 


Jl 


0.00     1.00    2.00    3  00    4.00    5.00    6.00    7.00    8.00  9.00 
Log,0(Count+1) 

Percent  Attar  Chill  Tank 


0.00    1.00    2.00    3.00    4.00    5.00    6.00    7.00    8.00  9.00 
Log,0(Count  +  1) 


Percent 


Attar  Chill  Tank 


60 


40  • 


Mm 


100 


O.OO    1.00    2.00    3.00    4.00    5.00    6  00    7  00    8  00  9.00 
Log,0(Count  +  1) 


0.00    1.00    2.00    3.00    4.00    5.00    6.00    7.00    8.00  BOO 
Log,0(Count+l) 


Figure  34.  Coliform  Distributions  -  Plant  PS3 
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Figure  35.  E.  coli  Distributions  -  Plant  PS3 
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Figure  36.  S.  aureus  Distributions  -  Plant  PS3 
Phase  1  Phase  3 
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Figure  37.  GFA  Distributions  -  Plant  PS3 
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Figure  38.  Salmonella  Distributions  -  Plant  PS3 
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Run  charts  for  the  six  quantitative  microbiology  factors  are  presented  in  Figure  39. 
There  were  sufficient  positive  results  to  compute  a  centerline  (CL)  from  Phase  I  data  for 
APC  @  35°C  ,  coliforms,  E.  coli,  and  GFAs.    There  was  a  run  of  points  above  the 
centerline  for  APC  @  35°C  and  coliforms  in  Phase  I  (starting  at  day  17). 

Percent  positive  Salmonella  was  proposed  to  be  used  for  process  control 
verification;  therefore,  percent  positives  for  Phases  I  and  III  were  compared.  The 
difference,  18.2  percent  vs  2.0  percent,  was  found  to  be  statistically  significant  (P<.05). 
Figure  38  and  Table  44  show  that  Salmonella  results  for  Phase  III  tended  to  be  lower  than 
those  for  Phase  I.  This  difference  was  to  be  expected,  since  the  Salmonella  results  for  the 
incoming  birds  in  Phase  III  tended  to  be  lower  than  those  in  Phase  I.  The  median 
chlorine  levels  were  10  ppm  in  both  Phase  I  and  Phase  III  and  has  no  explanatory  value. 

In  summary,  the  HACCP  microbiological  assessment  in  Poultry  Slaughter  Plant  PS 
3  showed  a  reduction  in  Salmonella  and  process  instability  in  Phase  I  for  some 
microbiology  factors.  The  microbiological  profile  of  the  product  produced  during  both 
phases  was  indicative  of  good  manufacturing  procedures. 
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Figure  39.  Run  Charts  for  Microbiology  Data 
Poultry  Slaughter  Plant  3  -  After  Chill  Tank 
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Figure  39  (continued).  Run  Charts  for  Microbiology  Data 
Poultry  Slaughter  Plant  3  -  After  Chill  Tank 
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Physical  Factors  Analysis: 

The  intended  length  of  the  HACCP  study  was  13  weeks  each  for  Phase  I  and 
Phase  III  (see  Methodology,  Chapter  2).  Phase  II  was  intended  to  be  a  transition  period 
into  HACCP  plan  implementation.  Table  45  displays  the  actual  length  of  the  study,  in 
weeks,  for  each  pilot  plant. 

TABLE  45:  LENGTH  OF  TIME  IN  WEEKS  FOR  PHASES 
I  AND  HI  PILOT  TESTING  FOR  PHYSICAL  FACTORS 
AT  THE  NINE  VOLUNTEER  PLANTS 


PLANT 

PHASE  I 

PHASE  m 

RF  1 

25 

27 

RF  2 

19 

26 

RF  3 

29 

29 

CS  1 

15 

25 

CS  2 

36 

26 

CS  3 

21 

26 

PS  1 

13 

26 

PS  2 

13 

27 

PS  3 

20 

26 

Variations  in  Observations: 

The  number  of  observations  for  each  data  collection  task  varied  within  a  plant 
between  both  Phases,  and  from  plant  to  plant  within  each  product  area.  Several  factors 
contribute  to  this  variation,  including:  the  necessity  of  dual-level  scheduling  rates  for 
verification  tasks  (three  times  a  week  or  once  every  2  weeks);  varied  length  of  the  study 
for  each  pilot  plant  and  phase,  which  ranged  from  13  weeks  to  36  weeks;  changes  in  plant 
monitoring  plans  of  the  pilot  plants;  variation  in  interpretation  of  data  collector 
instructions  detailing  when  to  perform  "each  occurrence"  tasks;  and  variation  in  HACCP 
product  availability  and  plant  operations. 

Presentation  of  Results: 

Plant  control: 

A  single  table  for  each  pilot  plant  shows  qualitatively  whether  that  plant  was 
following  an  established  procedure  and/or  documenting  actions  at  each  control  point  in 
each  of  the  physical  factors.  "Yes"  means  that  an  established  procedure  was  present  and 
being  followed  or  that  findings  were  being  documented  for  that  particular  control  point. 
"No"  means  that  an  established  procedure  was  not  present  or  that  results  were  not  being 
documented  for  that  particular  control  point. 
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When  corrective  action  was  observed  by  the  data  collector,  he/she  determined 
whether  the  action  taken  by  plant  personnel  was  appropriate.  For  example,  if  the 
corrective  action  result  is  3/4,  that  means  that  the  data  collector  observed  corrective  action 
being  taken  four  times  and  three  of  the  four  times  the  action  taken  was  appropriate. 

Agency  verification: 

The  Agency  verification  measurement  is  presented  together  in  one  table  for  each 
product  type.  The  results  of  the  verification  data  collection  tasks  are  presented  in  percent 
compliance  and  is  calculated  by  dividing  the  number  of  yes  responses  by  the  total  number 
of  observations  for  the  control  point.  For  example,  if  the  verification  results  for  control  of 
time/temperature  of  heating  product  is  recorded  as  100  (30  of  30),  then  that  indicates  that 
the  Agency  verification  for  heating  product  is  100  percent.  The  data  collector  measured 
the  actual  time/temperature  of  heating  a  product  30  times  and  all  30  observation  were  in 
compliance. 

Refrigerated  Foods  Plants: 

Plant  Control  -  Refrigerated  Foods  plant  1  (RF  1) 

Established  Procedures 

Prior  to  HACCP  implementation,  RF  1  had  established  procedures  for  all  but  one 
control  point:  instrument  calibration  (see  Table  46).  The  plant  established  a  procedure  for 
calibration. 

Documentation 

As  seen  in  Table  46,  for  Phase  I,  RF  1  documents  findings  for  all  control  points 
except  two:  instrument  calibration  and  heated  product  flow.  After  HACCP  was 
implemented,  the  plant  documented  instrument  calibration  results,  but  not  heated  product 
flow. 

Corrective  Action 

Data  collection  personnel  in  RF  1  observed  the  plant  taking  corrective  action  69 
times  during  Phase  I  and  58  times  during  Phase  III.  In  every  case,  it  was  the  appropriate 
corrective  action  for  a  score  of  100  percent. 
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TABLE  46:  RF  1  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


1  X 

PLANT  CONTROL 

Refrigerated  ] 

-ood  Plant  #  1 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE 

control  point 

FINDINGS  & 

ACTION* 

PHASE 

PHASE 

PHASE 

I 

rn 

I 

m 

I 

ru 

TEMPERATURE 

raw  meal  receiving  storage 

YES 

YES 

YES 

YES 

1/1 

area 

YES 

YES 

YES 

YES 

finished  product  area 

YES 

YES 

YES 

YES 

FOR  MAT/ORG  WHOL 

raw  meat  receiving 

YES 

YES 

YES 

YES 

non-meat  receiving 

YES 

YES 

YES 

YES 

6/6 

finished  product 

YES 

YES 

YES 

YES 

1/1 

TIME/TEMPERATURE 

heating 

YES 

YES 

YES 

YES 

2/2 

cooling 

YES 

YES 

YES 

YES 

3/3 

instrument  calib 

NO 

YES 

NO 

YES 

20/20 

1/1 

CROSS- 

CONTAMTNATION 

YES 

YES 

YES 

YES 

1/1 

employee  hygiene 

YES 

YES 

NO 

NO 

33/33 

47/47 

heat  product  flow 

PACKAGING 

mad  acceptable 

YES 

YES 

YES 

YES 

3/3 

2/2 

seal  integrity 

YES 

YES 

YES 

YES 

4/4 

modified  atmos 

YES 

YES 

YES 

YES 

1/1 

2/2 

hdlg  stmt/ dt  cde 

YES 

YES 

YES 

YES 

1/1 

(@)       If  YES  -  established  procedure  is  present;  If  NO  -  no  established  procedure  present 


(&)       If  YES  -  documents  findings;  If  NO  -  does  not  document  findings 

(#)        number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action 
was  observed 

Appropriate  corrective  action  rate: 

Phase  I  Phase  III 

1 00%  (69/69)      1 00%  (5 8/58) 
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Plant  Control  -  Refrigerated  Foods  plant  2  (RF  2) 

Established  Procedures 

Table  47  displays  RF  2's  results.  In  Phase  I,  RF  2  had  established  procedures  for 
9  of  15  control  points.  The  plant  did  not  add  more  procedures  after  HACCP 
implementation. 

Documentation 

Phase  I  data  show  RF  2  documented  findings  for  4  of  15  control  points.  After 
HACCP  implementation,  documentation  was  added  for  4  more  control  points.  The  plant 
did  not  document  findings  for  7  control  points  in  Phase  III. 

Corrective  Action 

The  data  collector  did  not  observe  RF  2  taking  corrective  action  during  Phase  I. 
The  plant  was  observed  taking  corrective  action  three  times  during  Phase  III,  but  none  of 
the  actions  was  recorded  as  appropriate. 
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TABLE  47:  RF  2  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  C 
Refrigerated 

ONTROL 
Food  Plant  #  2 

PHYSICAL  FACTOR/ 
control  point 

EST.  PROC.  PRESENT  @ 
PHASE 

DOCUMENTS 
FINDINGS  ic 

PHASE 

CORRECTIVE 
ACTION* 

PHASE 

I 

in 

I 

m 

I 

m 

TEMPERATURE 

raw  meat  receiving 
raw  meat  storage 
storage  area 
finished  product  area 

NO 
NO 
YES 

VTC 
I  LJ 

NO 
NO 
YES 

1  Lj 

NO 
NO 
YES 

I  CO 

NO 
NO 
YES 
I  Co 

FOR  MAT/ORG  WHOL 
raw  meat  receiving 
non-meat  receiving 
product  prep 

YES 
YES 
NO 

YES 
YES 
NO 

NO 
NO 
NO 

YES 
YES 
NO 

0/1 

TIME/TEMPERATURE  j 

cooking  i 
cooling 

instrument  calib 

YES 
YES 
NO 

YES 
YES 

NO 

YES 
YES 
NO 

YES 
YES 
NO 

CROSS- 
CONTAMINATION 
employee  hygiene 
heat  product  flow 

YES 
NO 

YES 
NO 

NO 
NO 

YES 
NO 

0/1 

PACKAGING 
mad  acceptable 
seal  integrity 
hdlg  stmt/ dt  cde 

YES 
NO 
YES 

YES 
NO 
YES 

NO 
NO 
NO 

YES 
NO 
NO 

0/1 

(@)       If  YES  -  established  procedure  is  present;  If  NO  -  no  established  procedure  present 


(&)       If  YES  -  documents  findings;  If  NO  -  does  not  document  findings 

(#)        number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action 
was  observed 

Appropriate  corrective  action  rate: 

Phase  I  Phase  III 
  0%  (0/3) 
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Plant  Control  -  Refrigerated  Foods  plant  3  (RF  3) 

The  data  collector  recorded  "no"  for  all  tasks  performed  during  a  2-week  period  in  Phase 
III  for  RF  3  and  commented  that  "key  plant  QC  personnel  on  vacation  or  jury  duty." 
Since  the 'data  collector  could  not  accurately  measure  plant  control  in  the  absence  of  key 
plant  personnel,  the  data  collection  results  were  changed  to  "not-performed"  for  this 
period. 

Established  Procedures 

Baseline  data  for  RF  3  showed  the  plant  had  established  procedures  in  place  for  all 
but  two  control  points  (Table  48).  After  implementing  its  HACCP  plan,  the  plant  had 
established  procedures  at  all  control  points.  The  established  procedure  for  detection  of 
foreign  material  in  non-meat  ingredients  was  strengthened  after  implementation  of  HACCP 
and  reduced  the  number  of  dental  claims  against  the  company. 

Documentation 

RF  3  documented  12  of  15  control  points  during  the  Baseline  Phase.  The  plant 
increased  documentation  to  14  control  points  in  Phase  III.  They  did  not  document  heat 
product  flow  under  the  cross-contamination  physical  factor. 

Corrective  Action 

During  Phase  I,  RF  3  was  observed  taking  appropriate  corrective  action  99  percent 
of  the  time  (460/464).  This  rate  improved  to  100  percent  in  Phase  III  (292/292). 
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TABLE  48:  RF  3  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  CONTROL 

Refrigerated 

Food  Plant  §  3 

DUVCIPAT   C  A  /  •'  T  Y~\ P  / 
rniilLAl.  f  AC  1  UR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  & 

PHASE 

PHASE 

PHASE 

I  I 

m 

I 

m 

I 

m 

TFMPFR  ATI  FRF 

raw  meat  receiving 

YES 

YES 

YES 

YES 

28/29 

18/18 

raw  meat  storage 

YES 

YES 

vrc 
I  to 

VTC 

I  to 

J  J/ JJ 

1  0  /I  D 
18/  IB 

storage  area 

YES 

YES 

1  no 

I  Co 

finished  product  area 

YES 

YES 

1  EO 

vrc 
i  no 

FOR  MAT/ORG  WHOL 

raw  meat  receiving 

VPS 
1  CO 

YFS 

1  CO 

YES 

YES 

32/32 

19/19 

non-meat  receiving 

YES 

YES 

NO 

YES 

30/30  . 

17/17 

finished  product 

NO 

YES 

NO 

YES 

18/18 

TIME/TEMPERATURE 

heating 

YES 

YES 

YES 

YES 

35/35 

21/21 

cooling 

YES 

YES 

YES 

YES 

58/59 

26/26 

instrument  calib 

YES 

YES 

YES 

YES 

55/56 

36/36 

CROSS- 

CONTAMINATION 

YES 

YES 

YES 

YES 

97/98 

49/49 

employee  hygiene 

NO 

YES 

NO 

NO 

19/19 

heat  product  flow 

PACKAGING 

mad  acceptable 

YES 

YES 

YES 

YES 

30/30 

19/19 

seal  integrity 

YES 

YES 

YES 

YES 

31/31 

19/19 

hdlg  stmt/dt  cde 

YES 

YES 

YES 

YES 

31/31 

14/14 

(@)       If  YES  -  established  procedure  is  present;  If  NO  -  no  established  procedure  present 


(&)       If  YES  -  documents  findings;  If  NO  -  does  not  document  findings 

(#)        number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action 
was  observed 


Appropriate  corrective  action  rate: 

Phase  I  Phase  III 

99%  (460/464)    100%  (292/292) 
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Agency  Verification  -  RF  Plants 

Control  Point  Verification  Rates 

The  total  Agency  verification  rate  for  the  control  points  ranged  from  91.0  percent  for 
the  control  point  covering  employee  hygiene  in  Phase  III  to  100  percent  for  1 1  other  control 
points  (Table  49).  The  control  points  focused  on  employee  hygiene  and  foreign 
material/organoleptic  wholesomeness  of  finished  product  showed  the  lowest  verification  rate 
at  93.3  percent. 

Pilot  Plant  Verification  Rates 

The  Agency  verification  rates  for  the  individual  pilot  plants  ranged  from  96.3  percent 
for  RF  1  during  Phase  I  to  98.9  percent  for  RF  1  in  Phase  III  (Table  49).  The  Agency 
verification  rate  increased  slightly  for  RF  1  and  RF  3  after  HACCP  implementation  and 
decreased  slightly  for  RF  2. 
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TABLE  49:  NUMBER  AND  PERCENTAGE  OF  OBSERVATIONS  FOR  AGENCY 
VERIFICATION  OF  PHYSICAL  FACTORS  FOR  RF  1,  RF  2,  and  RF  3 


AGENCY  VERIFICATION 


Refrigerated  Food  Plants 


PHYSICAL 

RF  t 

RF  2 

RF  3 

CONTROL  FT 

FACTOR/ 

TOTAL 

control  point 

PHASE  # 

PHASE* 

PHASE* 

PHASE  t 

1 

IB 

1 

'  m 

I 

:  rn 

[ 

eh 

TEMPERATURE 

meat  recv 

100 

90 

91 

100 

96 

100 

95.0 

97.5 

10/10 

9/10 

20/22 

25/25 

27/28 

5/5 

57/60 

39/40 

oieai  stor 

MM) 

NIP 

XML* 

NIP 

100 

100 

100 

100 

100 

too 

62/62 

56/56 

37/37 

4/4 

99/99 

60/60 

st  or  area 

94 

100 

100 

100 

100 

100 

98.4 

100 

15/16 

17/17 

15/15 

36/36 

32/32 

7/7 

62/63 

60/60  : 

fin  prod 

100 

100 

100 

100 

100 

100 

ICO 

too 

14/14 

19/19 

26/26 

32/32 

21/21 

7/7 

61/61 

5S/5S 

FOR  MT/ORG 

meat  recv 

100 

100 

100 

100 

100 

100 

ICO 

100 

7/7 

16/16 

5/5 

3/3 

26/26 

6/6 

38/38 

25/25 

non-mc  recv 

86 

100 

100 

100 

100 

100 

97.9 

100 

• 

7/8 

17/17 

13/13 

17/17 

28/28 

4/4 

48/49 

38/38 

prod  prep 

NIP 

NIP 

100 

100 

NIP 

NIP 

100 

100 

52/52 

48/48 

!  52/52 

48/48 

fin  prod 

100 

100 

NIP 

NIP 

99 

87 

7,3. .J 

12/12 

15/15 

77/78 

13/15 

89/90 

2E/30 

TIME/TEMP 

heatmg 

100 

100 

100 

100 

100 

100 

130 

too 

12/12 

10/10 

15/15 

26/26 

27/27 

15/15 

54/54 

51/51 

cooling 

100 

100 

100 

100 

100 

100 

100 

100 

22/22 

31/31 

41/41 

56/56 

57/57 

15/15 

120/120 

102/102 

mix  rum 

94 

97 

99 

97 

97 

100 

98.9 

98.* 

calib 

118/125 

15/15 

69/70 

33/34 

77/79 

25/25 

89/90 

73/73  \ 

CROSS-CON 

emp  hyg 

97 

95 

94 

88 

89 

98 

93  J 

91.0 

37/38 

41/43 

1 1 9/ 1 ?fl 

A.H/A  1 

207/222  ■■ 

:  193/212 

prod  flew 

100 

100 

100 

98 

99 

94 

99-2 

97.7 

12/12 

15/15 

54/54 

54/55 

69/70 

16/17 

135/136 

85/S7 

PACKAGING 

m*ti  accept 

94 

100 

100 

100 

88 

100 

93.4 

100 

15/16 

11/11 

20/20 

9/9 

22/25 

11/11 

57/61  . 

31/31 

seaJ  jntgt 

91 

100 

100 

100 

95 

100 

97.7 

100 

10/11 

14/14 

57/57 

58/58 

21/22 

11/11 

88/90 

S3/S3 

mod  atmos 

100 

100 

NIP 

NIP 

NIP 

NIP 

120 

too 

13/13 

16/16 

13/13 

16/16 

hdlg 

100 

100 

100 

90 

100 

100 

100 

94.5 

stmt/dt 

10/10 

15/15 

14/14 

28/31 

31/31 

9/9 

55/55 

52/55 

PLANT 

96.3 

98.9 

97.8 

96.4 

97.6  . 

9?.* 

TOTALS 

314/326 

263/264 

593/614 

602/617 

188/192 

NIP  •  not  in  HACCP  plan       (*)  %  yes  #yes/#observations 
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Cooked  Sausage  Plants: 

Plant  Control  -  Cooked  Sausage  plant  1  (CS  1) 

Established  Procedures 

Observations  during  Phase  I  (Table  50)  showed  that  CS  1  had  established 
procedures  in  place  for  14  of  17  control  points.  After  HACCP  implementation,  16  of  17 
control  points  had  established  procedures.  The  control  point  focusing  on  heated  product 
flow  did  not  have  an  established  procedure. 

Documentation 

Prior  to  HACCP  implementation,  CS  1  documented  findings  for  11  of  1 7  control 
points.  During  Phase  III,  the  plant  documented  16  of  17  control  points.  The  control  point 
focusing  on  heated  product  flow  was  not  documented  since  it  did  not  have  an  established 
procedure. 

Corrective  Action 

The  data  collector  observed  CS  1  taking  corrective  action  one  time  during  Phase  I, 
and  the  action  was  appropriate.  During  Phase  III,  CS  1  was  observed  taking  corrective 
action  four  times,  and  on  two  occasions  the  action  was  not  appropriate. 

Plant  Control  -  Cooked  Sausage  plant  (CS  2) 

Established  Procedures 

As  seen  in  Table  51,  CS  2  had  all  its  18  control  points  covered  by  established 
procedures,  both  before  and  after  HACCP  implementation. 

Documentation 

During  Phase  I,  CS  2  documented  findings  for  12  of  18  control  points.  However, 
after  HACCP  implementation  no  control  points  were  being  documented. 

Corrective  Action 

The  data  collector  observed  CS  2  taking  corrective  action  23  times  during  Phase  I, 
and  the  data  was  appropriate  in  22  cases.  During  Phase  III  the  data  collector  did  not 
observe  CS1  taking  corrective  action. 
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TABLE  50:  CS  1  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  CONTROL 

Cooked  Sausag 

e  Food  Plant  #\ 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  & 

# 

PHASE 

PHASE 

PHASE 

I 

m 

I 

in 

I 

m 

raw  meat  receiving 

YES 

YES 

YES 

YES 

raw  meat  storage 

NO 

YES 

NO 

its 

storage  area 

YES 

YES 

I  to 

vrc 
I  to 

product  area 

NO 

YES 

NO 

YES 

fin  prod  area 

YES 

YES 

YES 

YES 

FOR  MAT/ORG  WHOL 

raw  meat  receiving 

YES 

YES 

I  to 

I  to 

rework 

YES 

YES 

YES 

YES 

product  prep 

YES 

YES 

YES 

YES 

finished  product 

NO 

YES 

NO 

YES 

0/1 

REST  INGRED  CNTL 

formulation  ] 

YES 

YES 

YES 

YES 

TIME/TEMP 

heating 

YES 

YES 

YES 

YES 

cooling 

YES 

YES 

YES 

YES 

instrument  calib 

YES 

YES 

YES 

YES 

CROSS- 

CONTAMINATION 

YES 

YES 

YES 

YES 

1/1 

GiJlUlVf  CC    1 I  f  Civ  Jlw 

NO 

NO 

NO 

NO 

,  heat  product  flow 

PACKAGING 

mad  acceptable 

NO 

YES 

NO 

YES 

hdlg  stmt/dt  cde 

YES 

YES 

NO 

YES 

1/1 

1/2 

(@)         If  YES  -  established  procedure  present;  If  NO  -  no  established  procedure  present 


(&)         If  YES  •  documents  results;  If  NO  •  does  not  document  results 

(#)  number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action  was  observed 

Appropriate  corrective  action  rate: 

Phase  I  Phase  III 

100%  (1/1)  50%  (2/4) 
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TABLE  51:  CS  2  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  CONTROL 

Cooked  Sau sag 

e  Food  Plant  12 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  & 

PHASE 

PHASE 

PHASE 

I 

III 

I 

III 

I 

m 

TEMPERATURE 

raw  meat  receiving 

YES 

YES 

YES 

VIA 

NO 

raw  meat  storage 

YES 

YES 

NO 

NO 

1/1 

storage  ares 

YES 

YES 

NO 

NO 

nranurt  ar**a 

L/l  vUUbl  (Llva 

YES 

YES 

NO 

NO 

Tin  n rnn  arpa 

1111   Ul  UU  Cd 

YES 

YES 

NO 

NO 

FOR  MAT/ORG  WHOL 

raw  meat  receiving 

'  YES 

YES 

YES 

NO 

1/1 

nnn.mtir    r — i  n  n  n 
nUtJ-IDCifl  ItLVUIt 

YES 

YES 

YES 

NO 

rework 

YES 

YES 

YES 

NO 

1/1 

product  prep 

YES 

YES 

YES 

NO 

2/2 

finished  product 

VT7C 
I  to 

I  to 

NO 

NO 

3/3 

REST  INGRED  CNTL 

formulation 

YES 

YES 

NO 

1/1 

TIME/TEMP 

Heating 

YES 

YES 

NO 

NO 

.cooling 

YES 

YES 

NO 

NO 

instrument  calib 

YES 

YES 

NO 

NO 

CROSS- 

CONTAMINATION 

YES 

YES 

NO 

NO 

7/7 

employee  hygiene 

YES 

YES 

NO 

NO 

1/1 

heat  product  flow 

PACKAGING 

mat]  acceptable 

YES 

YES 

NO 

NO 

4/4 

hdlg  stmt/dt  cde 

YES 

YES 

YES 

NO 

1/2 

(@)         If  YES  -  established  procedure  present,  If  NO  -  no  established  procedure  present 


(&)         If  YES  -  documents  results;  If  NO  -  does  not  document  results 

(#)  number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action  was  observed 

Appropriate  corrective  action  rate: 
Phase  1  Phase  HI 

96%  (22/23)  — 
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Plant  Control  -  Cooked  Sausage  plant  3  (CS  3) 

The  data  in  Table  52  showed  that,  prior  to  HACCP  implementation,  12  of  15 
control  points  were  covered  by  established  procedures.  After  HACCP  implementation, 
only  the  control  point  covering  instrument  calibration  was  not  covered  by  established 
procedures. 

Documentation 

Prior  to  HACCP  implementation,  CS  3  documented  findings  for  6  of  1 5  control 
points.  After  HACCP  implementation,  only  one  control  point  —  instrument  calibration  — 
was  not  documented. 

Corrective  Action 

The  data  collector  observed  CS  3  taking  corrective  action  six  times  during  Phase  I; 
the  corrective  action  was  appropriate  in  three  instances.  During  Phase  III,  CS  3  took 
corrective  action  three  times;  on  all  occasions  none  of  the  actions  was  appropriate. 

Agency  Verification  -  CS  Plants 

Control  Point  Verification  Rates 

As  seen  in  Table  53,  the  total  Agency  verification  rate  for  the  control  points  ranged 
from  95.0  percent  to  100  percent.  The  control  point  focusing  on  foreign 
material/organoleptic  wholesomeness  of  finished  product  showed  the  lowest  verification 
rate  at  95.0  percent  in  Phase  III.  The  control  point  focusing  on  employee  hygiene  showed 
the  lowest  total  Agency  verification  rate  at  95.4  percent  in  Phase  I  and  97.5  percent  in 
Phase  HI. 

Pilot  Plant  Verification  Rates 

The  Agency  verification  rate  for  the  individual  pilot  plants  ranged  from  94.2 
percent  for  CS  2  in  Phase  I  to  99.8  percent  for  CS  3  in  Phase  III.  The  Agency 
verification  rate  increased  for  CS  2  and  CS  3  after  HACCP  implementation  and  decreased 
slightly  for  CS  1 . 
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TABLE  52:  CS  3  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  CONTROL 

Cooked  Sausag 

e  Food  Plant  #3 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  Be 

# 

PHASE 

PHASE 

PHASE 

I 

in 

T 
1 

LU 

1 

m 
HI 

TEMPERATURE 

raw  meat  receiving 

NO 

YES 

NO 

YES 

0/2 

storage  area 

YES 

YES 

YES 

YES 

fin  prod  area 

YES 

YES 

YES 

YES 

FOR  MAT/ORG  WHOL 

raw  meat  receiving 

YES 

YES 

YES 

YES 

1/1 

non-meat  reeving 

YES 

YES 

NO 

YES 

product  prep 

YES 

YES 

NO 

YES 

0/1 

finished  product  j 

YES 

YES 

YES 

YES 

0/1 

REST  INGRED  CNTL 

formulation 

YES 

YES 

NO 

YES 

1/1 

TIME/TEMP 

heating 

YES 

YES 

YES 

YES 

cooling 

NO 

YES 

NO 

YES 

instrument  calib 

NO 

NO 

NO 

NO 

CROSS- 

CONTAMINATION 

YES 

YES 

NO 

YES 

0/2 

employee  hygiene 

YES 

YES 

NO 

YES 

heat  product  flow 

PACKAGING 

mad  acceptable 

NO 

YES 

NO 

YES 

hdlg  stmt/dt  cde 

YES 

YES  ! 

YES 

YES 

1/1 

(@)         If  YES  -  established  procedure  present;  If  NO  -  no  established  procedure  present 


(&)         If  YES  -  documents  results;  If  NO  -  does  not  document  results 

(#)  number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action  was  observed 

Appropriate  corrective  action  rate: 

Phase  I  Phase  HI 

50%  (3/6)  0%  (0/3) 
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TABLE  53:  NUMBER  AND  PERCENTAGE  OF  OBSERVATIONS  FOR  AGENCY 
VERIFICATION  OF  PHYSICAL  FACTORS  FOR  CS  1,  CS  2,  and  CS  3 


AGENCY  VERIFICATION 

Cooked  Sausage  Food  PI 

ants 

PHYSICAL 

CS  J 

CS  2 

CS  3 

CONTROL  PT 

FACTOR/ 

TOTAL 

control  pain 

t               PHASE  if 

PHASE  S 

PHASE  4 

PHASE  # 

I 

IS 

Ms : 

m 

■  i 

m 

I 

is 

TEMPERATU 

R.F 

raw  meat  recv 

100 

100 

83 

100 

100 

98 

100 

11/11 

12/12 

5/6 

16/16 

35/35 

0/0 

51/52 

28/2/  ; 

raw  meat  star 

100 

100 

93 

100 

NIP 

NIP 

98.2 

10c 

44/44 

44/44 

13/14 

15/15 

57/58 

59/59  : 

storage 

100 

96 

100 

100 

100 

100 

10O 

'98.5 

n/n 

22/23 

7/7 

20/20 

38/38 

24/24 

58/53 

66/67  ; 

product  prep 

100 

100 

92 

100 

100 

100 

99.0 

100 

42/42 

90/90 

12/13 

16/16 

49/49 

62/62 

103/104 

168/16? 

us  prod  zits 

100 

100 

80 

100 

100 

100 

98.6 

100 

14/14 

20/20 

4/5 

24/24 

56/56 

76/76 

74/75 

120/120 

FOR  MAT/OI 

IG 

raw  meat  recv 

100 

100 

100 

100 

100 

100 

too 

11/11 

14/14 

11/11 

13/13 

10/10 

18/  18 

7  A  /"7  A 

24/24 

:    40/40  ■■■ 

51/51 

Don-tnezr  rec 

NIP 

NIP 

88 

1 00 

100 

100 

974 

too 

7  /O 

7/8 

12/12 

17/17 

3Z/37 

n  a  /l  a 
24/  24 

44/45 

;  36/36 

rework 

100 

100 

4  AA 

100 

100 

TvfTD 

INlr 

TvTTD 

100 

100 

14/ 14 

14/14 

O/O 
0/6 

11  2 

22/22 

16/16 

product  prep 

100 

95 

100 

100 

100 

98 

100 

97,6 

20/20 

19/20 

11/11 

11/11 

46/46 

58/59 

77/77 

88/90 

finished  produa 

>  100 

90 

83 

91 

100 

100 

99.0 

95.0 

46/46 

45/50 

■  5/6 

10/11 

61/61 

61/61 

:  100/101 

116/122 

RESTING 

100 

100 

100 

100 

Q  O 

70 

1  AA 
100 

98.7  . 

100 

CNTL 

1 7  /1  o 

12/12 

11/11 

11/11 

1/  /l  y 
16/  lb 

17/17 

1//  1/ 

48/47 

oU/  00 

4 

!S/88 

formulation 

TIME/ TEMP 

beating 

\  aa 

100 

1  aa 
100 

100 

*  aa 

100 

1  AA 
100 

100 

100 

17/1  7 

17/17 

11/(1 
13/13 

7/7 

7/7 

13/13 

0 1  /i  1 
21/21 

40/40 

45/45 

66/66 

cooling 

100 

100 

100 

100 

98 

100 

98.5 

too 

16/18 

18/18 

6/6 

17/17 

40/41 

66/66 

64/65 

101/101: 

mstrumeiH 

100 

100 

100 

100 

100 

100 

100 

100 

18/18 

12/12 

10/10 

26/26 

5/5 

4/4 

33/33 

42/42 

CROSS-CON 

crop  hygiene 

94 

89 

89 

100 

97 

99 

95.4 

97.5 

48/51 

51/57 

26/29 

44/44 

132/136 

184/185 

206/216 

279/286  : 

heat  prod  {low 

100 

100 

100 

100 

100 

100 

10O 

100 

48/48 

43/43 

12/12 

18/18 

44/44 

60/60 

104/104 

121/121  ; 

Packaging 

mail  accept 

100 

100 

100 

100 

100 

100 

.100 

100 

45/45 

43/43 

2/2 

10/10 

47/47 

63/63 

94/94 

116/116  - 

ndlg  itmt/dt 

100 

98 

100 

100 

100 

100 

130 

n.7 

64/64 

76/76 

ede 

15/15 

82/84 

9/9 

17/17 

52/52 

163/165 

PLANT 

99.3 

97.4 

94.2 

99.7 

99.2 

994 

TOTALS 

447/45!) 

570/585 

179/190 

316/317 

762/768 

934/936 

NIP-not  in  HACCP  plan  (#)  %  yes  #yes/#obs 
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Poultry  Slaughter  Plants: 

Plant  Control  -  Poultry  Slaughter  plant  1  (PS  1) 

Established  Procedures 

Both  before  and  after  implementing  HACCP,  PS  1  had  established  procedures  in 
place  for  all  of  its  control  points,  as  shown  in  Table  54. 

Documentation 

Both  before  and  after  implementing  HACCP,  PS  1  documented  findings  for  all  of 
its  control  points.  Data  collection  results  show  that  the  control  points  covering  operational 
sanitation  and  scald  tank  overflow  were,  respectively,  accurately  documented  at  87  percent 
(21  of  24)  and  80  percent  (12  of  15),  of  the  time  in  Phase  I. 

Corrective  Action 

PS  1  was  observed  taking  the  appropriate  corrective  action  99  percent  of  the  time 
during  both  Phase  I  (357/360)  and  Phase  III  (757/761). 

Plant  Control  -  Poultry  Slaughter  plant  (PS  2) 

Established  Procedures 

As  shown  in  Table  55  PS  2  had  established  procedures  in  place  for  all  of  its 
control  points,  both  before  and  after  implementing  HACCP.  Data  collection  results  show 
that  PS  2  followed  its  established  procedures  69  percent  (411  of  600)  and  76  percent  (649 
of  853)  of  the  time  in  Phases  I  and  III,  respectively.  Established  procedures  covering  the 
control  points  for  operational  sanitation,  midshift  rinse,  viscera  equipment  sanitation,  and 
rehang  were  followed  less  than  50  percent  of  the  time  in  one  or  both  of  the  phases. 

Documentation 

Prior  to  HACCP  implementation,  PS  2  documented  findings  for  4  of  1 1  control 
points.  After  HACCP  implementation,  7  of  1 1  control  points  were  documented. 
Documentation  covering  the  control  points  for  scald  tank  overflow,  offline  reprocessing, 
and  drawing  viscera  were  added  as  a  result  of  HACCP  implementation. 

Corrective  Action 

PS  2  was  observed  taking  the  appropriate  corrective  action  63  percent  (251  of  401) 
and  37  percent  (73  of  195)  of  the  time  during  Phases  I  and  III,  respectively. 
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TABLE  54:  PS  1  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


■ 

PLANT  CONTROL 

Poultry  Slau| 

;hter  Plant  #1 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  & 

PHASE 

PHASE 

PHASE 

1 

m 

T 
1 

m 

T 
1 

m 

1X1 

SANITATION 

pre-ope  rational 

YES 

YES 

YES 

YES 

25/25 

35/35 

operational 

YES 

YES 

YES 

YES 

23/23 

47/48 

midshaft  rinse 

YES 

YES 

YES 

YES 

27/27 

44/46 

eviicena  equipment 

YES 

YES 

YES 

YES 

26/26 

51/51 

scald  tank  overfl 

YES 

YES 

YES 

YES 

14/15 

53/53 

PRODUCT  HANDLING 

Product  flow 

YES 

YES 

YES 

YES 

22/22 

45/45 

offline  salvage 

YES 

YES 

YES 

YES 

25/25 

46/46 

offline  reproc 

YES 

YES 

YES 

YES 

25/25 

42/42 

cndmd  mat  contri 

YES 

YES 

YES 

YES 

27/27 

51/51 

CONTAMINATION 

dressing/handling 

YES 

YES 

YES 

YES 

13/13 

52/52 

rehang 

YES 

YES 

YES 

YES 

25/26 

45/46 

drawing  visce 

YES 

YES 

YES 

YES 

26/26 

47/47 

pre-chiil 

YES 

YES 

YES 

YES 

26/26 

48/48 

post-chill 

YES 

YES 

YES 

YES 

23/24 

46/46 

salvage  contri 

YES 

YES 

YES 

YES 

30/30 

105/105 

(@)         If  YES  -  established  procedure  present,  If  NO  -  no  established  procedure  present 


(&)         If  YES  -  documents  results;  If  NO  -  does  not  document  results 

(#)  number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action  was  observed 

Appropriate  corrective  action  rate: 

Phase  1  Phase  III 

99%  (357/360)         99%  (757/761) 
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TABLE  55:  PS  2  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  CONTROL 

Poultry  Slauf 

fhter  Plant  #2 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  @ 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  & 

n 

PHASE 

PHASE 

PHASE 

T 

1 

m 
m 

I 

in 

I 

m 

SANITATION 

pre-  opera  tional 

YES 

YES 

YES 

YES 

19/34 

1/6 

operational 

YES* 

YES 

NO 

NO 

11/28 

0/9 

midshift  rinse 

YES* 

YES 

NO 

NO 

6/26 

1/9 

eviscena  equipment 

YES* 

YES* 

NO 

NO 

12/35 

2/19 

scald  tank  overfl  , 

YES 

YES 

NO 

YES 

12/15 

0/3 

PRnm  u — r  hantii  n\](~; 

Product  flow 

YES 

YES 

NO 

NO 

23/30 

0/9 

offline  salvage 

YES 

YES 

NO 

NO 

15/22 

0/3 

offline  reproc 

YES 

YES 

NO 

YES 

20/27 

20/23 

endmd  mat  contrl 

YES 

YES 

NO 

NO 

23/24 

0/1 

CONTAMINATION 

dressing/handling 

YES 

YES 

NO 

■  NO 

16/23 

2/18 

rehang 

YES* 

YES* 

NO 

NO 

13/35 

1/39 

drawing  visce  i 

YES 

YES 

NO 

YES 

19/28 

11/15 

pre-chill 

YES 

YES 

YES 

YES 

20/20 

12/15 

post-chill 

YES 

■  YES 

YES 

YES 

23/23 

3/4 

salvage  contrl 

YES 

YES 

YES 

YES 

19/26 

20/22 

If  YES  -  established  procedure  present;  If  NO  -  no  established  procedure  present 


(&)         If  YES  -  documents  results;  If  NO  -  does  not  document  results 

(#)  number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action  was  observed 

Appropriate  corrective  action  rate: 

Phase  I  Phase  III 

63%  (251/401)         37%  (73/195) 

(•)  Followed  Established  Procedure  <  50%  of  time 
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Plant  Control  -  Poultry  Slaughter  plant  3  (PS  3) 

Established  Procedures 

As  shown  in  Table  56,  prior  to  implementing  HACCP,  PS  3  had  established 
procedures  in  place  for  10  of  15  control  points.  After  implementing  HACCP,  PS  3  had 
established  procedures  for  12  of  15  control  points.  Established  procedures  covering  pre- 
operational sanitation  and  scald  tank  overflow  were  added  as  a  result  of  the  HACCP  plan. 

According  to  the  data  collection  results,  the  established  procedure  for  the  rehang 
control  point  was  dropped  in  Phase  III.  The  established  procedure  for  the  control  point  for 
evisceration  sanitation  was  followed  74.2  percent  (46  of  62)  of  the  time  during  Phase  I. 

Documentation 

Prior  to  implementing  HACCP,  PS  3  documented  findings  for  10  of  15  control  points. 
After  implementing  HACCP,  documentation  was  added  for  two  control  points  —  pre- 
operational sanitation  and  scald  tank  overflow.  According  to  the  data  collection  results, 
documentation  covering  rehang  was  dropped  in  Phase  III. 

Corrective  Action 

PS  3  was  observed  taking  the  appropriate  corrective  action  95  percent  (560  of  587) 
and  99  percent  (851  of  854)  of  the  time  during  Phases  I  and  III,  respectively. 

Agency  Verification  -  PS  Plants 

Control  Point  Verification  Rates 

The  total  Agency  verification  rate  for  the  control  points  ranged  from  19.5%  for  the 
drawing  viscera  in  Phase  III  to  95.2%  for  product  flow  in  Phase  III  (Table  57).  The  control 
point  for  drawing  viscera  showed  the  lowest  verification  rate  in  both  Phases  I  and  III,  at 
23.5%  and  19.5%,  respectively. 

Pilot  Plant  Verification  Rates 

The  Agency  verification  rate  for  the  individual  Poultry  Slaughter  plants  ranged  from 
36.5  percent  for  PS  2  in  Phase  I  to  86.8  percent  for  PS  1  in  Phase  III.  The  total  Agency 
verification  rate  increased  for  all  three  Poultry  Slaughter  plants  after  HACCP  implementation. 

Before  implementing  HACCP,  PS  2  documented  findings  for  4  of  1 1  control  points. 
After  HACCP  implementation,  7  of  1 1  control  points  were  being  documented. 
Documentation  covering  the  control  points  for  scald  tank  overflow,  offline  reprocessing,  and 
drawing  viscera  were  added  as  a  result  of  HACCP  implementation. 
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TABLE  56:  PS  3  OBSERVATION  RESULTS  OF  PLANT  CONTROL  FOR 
PHYSICAL  FACTORS 


PLANT  CONTROL 

Poultry  Slauf 

;hter  Plant  #3 

PHYSICAL  FACTOR/ 

EST.  PROC.  PRESENT  ® 

DOCUMENTS 

CORRECTIVE  ACTION 

control  point 

FINDINGS  & 

u 

PHASE 

PHASE 

PHASE 

I 

m 

I 

ffl 

I 

ffl 

SANITATION 

pre-operational 

NO 

YES 

NO 

YES 

57/58 

operational 

YES 

YES 

YES 

YES 

53/55 

70/70 

midshift  rinse 

YES 

YES 

YES 

YES 

57/61 

70/70 

eviscena  equipment 

YES 

YES 

YES 

YES 

46/61 

75/77 

scald  tank  overfl 

NO 

YES 

NO 

YES 

76/77 

PRODUCT  HANDLING 

Product  flow 

YES 

YES 

YES 

YES 

56/58 

73/73 

offline  salvage 

NO 

NO 

NO 

NO 

offline  reproc 

NO 

NO 

NO 

NO 

rnrifflH  mir  *~r\nr rl 

t  iuiiiiu   LUdl  UUilLIl 

YES 

YES 

YES 

YES 

56/57 

70/70 

CONTAMINATION 

dressing/ handling 

NO 

NO 

NO 

NO 

re  hang 

YES 

NO 

YES 

NO 

5/5 

drawing  visce 

YES 

YES 

YES 

YES 

53/53 

74/75 

pre-chill 

YES 

YES 

YES 

YES 

61/62 

70/70 

post -chill 

YES 

YES 

YES 

YES 

55/56 

70/70 

salvage  contrl 

YES 

YES 

YES 

YES 

118/119 

146/146 

(@)         If  YES  -  established  procedure  present;  If  NO  -  no  established  procedure  present 


(&)         If  YES  -  documents  results;  If  NO  -  does  not  document  results 

(tt)  number  of  times  appropriate  corrective  action  observed/total  number  of  times  corrective  action  was  observed 

Appropriate  corrective  action  rate: 

Phase  1  Phase  III 

95%  (560/587)         99%  (85 1/854) 
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TABLE  57:  NUMBER  AND  PERCENTAGE  OF  OBSERVATIONS  FOR  AGENCY  VERIFICATION  OF 
PHYSICAL  FACTORS  FOR  PS  I,  PS  2,  and  PS  3 


AGENCY  VERIFICATION 


Poultry  Slaughter  Plants 


PHYSICAL 

PS  1 

PS  2 

PS  3 

CONTROL  PT 

FACTOR/ 

TOTAL 

control  point 

PHASE 

PHASE 

PHASE 

PHASE 

i 

IB 

I 

IB 

I  1 

m 

I 

pr&*op6fattii 

80 

91 

0 

15 

64 

45 

54.9 

46.9 

4/3 

10/11 

0/11 

2/13 

35/55 

26/57 

39/71 

38/81 

i 

100 

100 

18 

50 

50 

93 

:::»i::'iqj':'::::>:;:::: 

7/7 

16/16 

2/11 

7/14 

8/16 

13/14 

17/34 

36/44 

nxdsuiir  nose  .  ■ 

82 

64 

22 

75 

46 

100 

51.5 

76.9 

9/11 

9/14 

2/9 

12/16 

6/13 

9/9 

17/33 

30/39  i 

75 

73 

7 

3 

69 

69 

55.9 

58.8 

12/16 

16/22 

1/14 

1/32 

20/29 

9/13 

33/59 

26/67  ;! 

79 

90 

54 

85 

78 

94 

90.2 

overflow 

26/33 

36/40 

22/41 

46/54 

42/54 

66/70 

90/128 

148/164  : 

riwuuu  ii\J  w 

100 

100 

82 

94 

100 

90 

93  J 

95.2 

Q  /O 
7/7 

14/14 

7/ ll 

17/10 

17/  Jo 

12/12 

9/10 

30/32 

40/42 

100 

100 

42 

46 

21 

58 

36.2 

62.7 

0/0 

16/16 

C  /1 1 
3/12 

6/ 13 

8/38 

42/73 

lt/58 

64/102  . 

100 

100 

77 

67 

14 

45 

40.9 

56.6 

11/11 

14/14 

10/13 

14/21 

6/42 

32/71 

27/66 

60/106 

*"nsst"  /*rt#*fJ  .;••;*'•-•: 

100 

100 

90 

91 

94 

83 

6/6 

15/15 

9/10 

20/22 

17/18 

10/12 

32/54 

45/49  | 

CONTAMINATION 

db-ess/iiaodliog 

78 

98 

13 

28 

91 

94 

66.4 

75.4 

28/36 

53/54 

5/36 

16/57 

54/59 

75/80 

87/131 

144/191 

re  bang 

89 

89 

4 

17 

60 

35 

47.9 

397  : 

8/9 

17/19 

1/21 

4/24 

26/43 

27/78 

3S/73 

♦8/121  i 

58 

38 

15 

6 

12 

15 

233 

19.5 

7/12 

11/29 

2/13 

2/33 

3/26 

3/20 

12/51 

16/82 

pre-ciull 

85 

69 

0 

15 

13 

5 

36.1 

27.3 

11/13 

11/16 

0/8 

3/20 

2/15 

1/19 

13/36 

15/55 

post-chtU 

100 

844 

95 

9 

27 

9 

78.0 

53,9 

11/11 

16/19 

18/19 

2/22 

3/11 

1/15 

32/4! 

19/56  ; 

salvgCTmtri 

73 

92 

16 

7 

44 

51 

59.1 

75,0 

52/71 

93/101 

2/12 

1/14 

14/32 

17/33 

68/115 

111/148 

PLANT  TOTALS 

81.0 

84-8 

36.S 

41.0 

55.3 

59.2 

209/258 

347/400 

88/241 

153/373 

256/463  % 

340/574 
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Physical  Factors  Summary: 

The  pilot  test  results  demonstrate  that  industry,  academia,  and  government  can 
cooperatively  develop  generic  HACCP  models  to  target  health  and  safety  issues.  Plant- 
specific  plans  may  be  developed  from  HACCP  models  and  applied  to  specific  meat  and 
poultry  product  categories.  The  plan  assures  that  plants  control  their  processes  to  manage 
critical  areas  of  food  product  manufacture.  Plants  control  their  processes  by  adhering  to 
approved  established  procedures,  properly  documenting  findings,  and  appropriately  correcting 
of  process  deviations  for  each  identified  process  control  point. 

Verification  testing  is  essential  to  assess  plant  control.  The  study  demonstrated  that 
simply  adhering  to  an  established  procedure  and  properly  documenting  findings  did  not 
guarantee  that  a  control  point  was  being  managed.  The  study  also  showed  that  having  an 
established  procedure  did  not  necessarily  mean  that  it  was  being  followed. 
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QUALITA  TTVE  PLANT  DA  TA : 


Assessment  of  Volunteer  Plants'  Management  Commitment  to  Control  Manufacturing 
Processes,  Quality,  and  Regulatory  Compliance 

As  part  of  the  overall  assessment  of  the  HACCP  pilot  test,  FSIS  gathered  data 
through  observations  to  determine  the  overall  regulatory  compliance  standards  of  the  nine 
volunteer  plants.  The  analysis  provides  an  overview  of  the  volunteer  plants'  environment, 
as  well  as  their  management  ability  and  willingness  to  control  the  manufacturing  processes 
and  their  attitudes  and  commitment  to  quality  and  regulatory  compliance  for  health  and 
safety.  Eleven  categories  (See  Chapter  2)  were  identified  and  scales  developed  to  gauge 
the  plants'  and  managements'  commitment  to  manufacturing  process  control,  quality,  and 
regulatory  compliance. 

Each  major  factor  had  a  number  of  subfactors  or  items.  The  maximum  number  of 
both  subfactors  and  items  is  112.  However,  not  all  items  apply  to  every  plant  given  the 
product  or  process  used  by  the  plant.  The  analysis  of  the  information  demonstrates  the 
impact  of  HACCP  on  the  general  functioning  of  the  plants.  Data  were  collected  during 
Phases  I  and  III  for  the  1 1  factors  (see  Table  58).  A  summary  of  the  plant's  results  is 
provided,  noting  when  the  change  occurred  from  Phase  I  to  Phase  III,  and  the  direction  of 
that  change  for  each  pilot  plant  by  process  type. 

FSIS  made  qualitative  assessments  by  observing  plant  activities  and  practices  and 
interviewing  plant  management  and  inspection  personnel.  Observations  were  recorded  on 
a  five  point  scale  --  high,  high  medium,  medium  medium,  low  medium  and  low.  In  the 
analysis,  these  ratings  were  converted  to  a  numerical  scoring  scheme  with  high  receiving  a 
score  of  one  and  low  receiving  a  score  of  five. 
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TABLE  58:  CATEGORIES,  NUMBER  OF  FACTORS,  SUBFACTORS  AND  TOTAL  NUMBER  OF 
ITEMS  RATED  FOR  THE  QUALITATIVE  ASSESSMENT 


CATEGORIES 

NUMBER  OF 
r  AC  1  UKi 

NUMBER  OF 
SUnr  AC  1  Oro 

TOTAL  NUMBER  OF 
ITEMS  RATED 

1,  Genera]  Housekeeping 

8 

8 

2.  Condition  of  Facilities 

9 

- 

9 

3.  Employee  Hygiene 

6 

- 

6 

4.  Employee  Sanitary  Practices 

8 

8 

5.  Employee  Attitudes  towards 
Inspection  Requirements 

2 

7 

7 

6.  Employee  Training 

2 

10 

10 

7.  Management  Supervisory 
Attitudes 

2 

9 

9 

8.  Management  Response  to 
Problem 

4 

12 

12 

9.  Management  Strategics  to 
Prevent  Contamination  Hazards 

2 

16 

16 

10.  Product  Quality 

2 

12 

12 

11.  Management  Programs  and 
Systems 

5 

15 

15 

TOTAL 

50 

81  ! 

112 
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Refrigerated  Foods  Plants: 

Refrigerated  Foods  plant  1  (RF  1): 

RF*1  was  rated  on  107  of  1 12  items  or  subfactors.  Table  59  presents  the  overall 
scoring  for  RF  1 .  The  table  highlights  what  occurred  between  Phases  I  and  III.  The 
diagonally  shaded  cells  represent  the  number(s)  of  items  on  which  no  change  occurred 
between  the  two  phases;  cells  to  the  right  of  the  shaded  area  portray  positive  change  (e.g., 
going  from  a  medium  medium  rate  of  3  to  -a  high  medium  rating  of"  2);  cells  to  the  left  of 
the  shaded  area  represent  a  negative  change  (e.g.,  going  from  a  medium  medium  rating  of 
3  to  a  low  medium  rating  of  4).  The  low  medium  score  of  4  indicates  the  plant  is  just 
barely  meeting  the  minimum  requirements  while  the  high  score  of  1  demonstrates  the 
plant  greatly  exceeds  the  requirements.  The  average  Phase  I  score  is  found  in  the  table  at 
BEFORE  AVG.  while  the  average  Phase  III  score  is  found  at  AFTER  AVG.  Below  each 
table  is  an  itemized  list  of  subfactors  that  changed  and  the  direction  of  change. 

RF  l's  average  score  for  Phase  I  was  2.49  which  is  between  medium  medium  and 
high  medium.  Its  Phase  III  score  was  2.46,  which  is  a  positive  change  from  Phase  I. 

Of  the  107  items  observed,  RF  1  had  changes  in  13  items  from  Phase  I  and  Phase 
III  observations.  There  were  five  ratings  going  from  medium  medium  to  high  medium, 
which  demonstrates  that  the  plant  exceeds  the  requirements.  For  three  items,  RF  1  moved 
from  high  medium  to  high.  In  these  cases,  the  initiatives  taken  by  RF  1  for  the  three 
factors  or  subfactors  greatly  exceeded  the  regulatory  requirements.  For  five  factors,  RF  1 
went  from  high  medium  to  medium  medium.  This  change  shows  that  the  plant  is  still 
soundly  meeting  the  requirements.  The  major  factor  areas  where  negative  movement 
occurred  were  in  General  Housekeeping  (1  item),  Condition  of  Facilities  (2  items),  Plant 
Employee  Sanitary  Practices  (1  item),  and  Plant  Management  Measures  to  Protect  Product 
from  Contamination  Hazards.  (See  Table  59  for  specific  items  which  showed  negative 
and  positive  changes). 

Refrigerated  Foods  plant  2  (RF  2): 

RF  2  was  rated  on  91  of  the  112  total  items.  Table  60  displays  the  scores  and  lists 
the  items  that  changed.  RF  2  went  from  a  Phase  I  average  score  of  2.73  to  a  Phase  III 
average  score  of  2.52.  These  scores  show  that,  overall,  the  plant  was  above  the  medium- 
medium  score  of  3  (meeting  the  regulatory  requirements)  and  moving  toward  exceeding 
the  requirements.  Of  the  91  items,  RF  2  had  positive  movement  in  19  items,  two  going 
from  low  medium  to  medium  medium,  1 5  going  from  medium  medium  to  high  medium 
and  two  moving  from  high  medium  to  high.  Product  Quality  and  Safety  had  a  total  of 
five  items,  out  of  the  nine  total,  shift  to  the  positive  items  for  this  factor.  General 
Housekeeping  included  the  next  highest  number  of  positive  changes,  with  three. 
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TABLE  59:  QUALITATIVE  EVALUATION  OF  REFRIGERATED  FOODS  PLANT 
NUMBER:  RF  1 


PHASE  I SQORE 

PHASE  m  SCORE  » 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

0 

0 

4 

2 

2 

8 

3 

52  . 

5 

57 

171 

2 

5 

31 

3 

39 

78 

1 

9 

0 

o 

SUM 

0 

2 

57 

36 

12 

107 

SCORE 

0 

8 

171 

72 

12 

263 

266 

BEFORE  AVG. 

2.49 

AFTER  AVG. 

2.46 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  59  (CONTINUED) 


FACTOR 

ITEM 
NUMBER 

DESCRIPTION 

SCORE 
CHANGE 

NEGATIVE  CHANGE 

I 

5 

Sanitation  or  cleanup  staff 

2  to  3 

n 

2 

Maintenance  of  loading  docks 

2  to  3 

n 

3 

Shipping/ receiving  areas 

2  to  3 

IV 

8 

Sanitary  storage  of  product  contact  items 

2  to  3 

K 

21 

Management  programs  prevent  contamination  from 
overhead 

2  to  3 

POSITIVE  CHANGE 

n 

9 

Plant  improvement  plan 

3  to  2 

m 

4 

Company  hygiene  policies 

2  to  1 

m 

5 

Visibility  of  policies 

2  to  1 

m 

6 

Scope  of  hygiene  policies 

2  to  1 

VI 

2a 

Employee  inplant  refresher  training  1 

3  to  2 

vn 

la 

Management  programs  encourage  and  use  employee 
suggestions 

3  to  2 

vn 

lb 

Uses  TQM 

3  to  2 

•  X 

la 

Employees  trained  to  take  independent  action 

3  to  2 
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TABLE  60:  QUALITATIVE  EVALUATION  OF  REFRIGERATED  FOODS  PLANT 
NUMBER:  RF  2 


PHASE  I  SCORE 

PHASE  m  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

0 

0 

4 

I 

2 

3 

12 

3 

15 

66 

198 

2 

2 

10 

] 

6 

6 

SUM 

0 

1 

53 

29 

8 

91 

SCORE 

0 

4 

159 

58 

8 

229 

248 

BEFORE  AVG. 

2.73 

AFTER  AVG. 

2.52 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  60  (CONTINUED) 


FACTOR 

ITEM 

X  TT  T~»  /Tl  T"T1 

NUMBER 

DESCRIPTION 

SCORE 
CHAJNCjE 

positive  CHANGE 

I 

2 

storage  or  pallets,  supplies 

4  to  3 

T 
J 

4 

Control  of  processing  debris 

J  to  z 

T 
1 

"5  -" 

Sanitation  or  cleanup  staff 

z  to  1 

TT 

u 

2 

Loading  docks  • 

3  to  Z 

TT 

u 

3 

shipping/  receiving  areas 

J  to  z 

TTT 
111 

1 

Employee  hand  washing 

J  to  z 

m 

5 

Visibility  of  policies 

4  to  3 

VI 

la 

Sanitary  personal  hygiene 

3  to  2 

VI 

lb 

Sanitary  product  handling 

3  to  2 

vn 

la 

Manappmpnt  nrnfffamc  ^firniiMCP  ann  n^p*  pmn  nvpp 

i»I  dJ  lid  t  Cl  ( ICLit   L/J  UgioiUJ  CUtUUIACC  oAlU    LUC  CJIJUIUVCC 

suggestions 

3  to  2 

vm 

vJ\  ana c»*» m f*v\ t  in i r*i n  ti/ npn  n rrt h  1  pm c     pi cp 

JVldJUlgClilCllL   dfLUUU    WJJCil   JL> !  U  UlCll  lis  4J  1_>C 

3  to  2 

IX 

51 
*J 

P rpvpnT  n roHi if*t  rnnti m inarirtp  ti/i t i-i  f n rpifffi 

X  ICVCllL   UI  UUULL  t-Ul         LiiiLdLiUl  1    WJLJJ  JLMCltJJ 

matenals 

3  rn  ? 

X 

lb 

Processing  procedures  designed  with  multiple  safety 
controls 

3  to  2 

X 

Id 

Adequate  safety  margin  built  in 

3  to  2 

X 

lg 

Management  keeps  inspection  informed 

2  to  1 

X 

2b 

Employees  take  independent  action 

3  to  2 

X 

2c 

Employees  actively  involved  in  production 
planning 

3  to  2 

XI 

2a 

Plant  follows  TQC  procedures 

3  to  2 

XI 

3 

Management  QA/QC  programs  ! 

3  to  2 
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Refrigerated  Foods  plant  3  (RF  3): 

Table  61  displays  the  scoring  for  RF  3  during  Phases  I  and  III  observations.  A 
total  of  105  items  out  of  112  were  scored.  RF  3's  average  scores  were  2.61  and  2.50  for 
Phase  I  and  Phase  III,  respectively.  Again,  these  averages  show  the  plant  was  above  the 
medium  medium  rating  before  operating  under  HACCP  and  moved  towards  a  rating  of 
high  medium  afterward.  RF  3  had  14  positive  changes  and  no  negative  changes.  Of  the 
14  positive  changes,  13  were  from  medium  medium  to  high  medium.  The  factor  with  the 
most  changes  was  Plant  Management  Response  to  Problems,  with  five  items  going  from 
medium  medium  to  high  medium. 

Refrigerated  Foods  Plants  Summary: 

As  seen  in  Table  62,  all  three  RF  plants  were  above  medium  medium  during  Phase 
I  and  increased  their  scores  during  Phase  III.  The  overall  Phase  I  average  was  2.61  and 
the  Phase  III  average  was  2.5.  Of  the  46  changes,  39  were  positive  with  33  items  moving 
from  medium  medium  to  high  medium.  When  the  data  from  all  the  Refrigerated  Foods 
plants  are  combined,  some  trends  can  be  identified.  Although  each  factor  category  shows 
at  least  one  change  in  a  subfactor,  the  factors  with  the  least  amount  of  change  are:  Plant 
Employee  Attitudes  about  Inspection;  Plant  Employee  Training;  Plant  Management 
Attitudes  Toward  Employee  Supervision;  and  Plant  Management  Programs  and  Systems. 
The  factors  showing  the  greatest  amount  of  change  are:  Product  Quality  and  Safety;  Plant 
Management  Response  to  Problems;  and  Plant  Measures  to  Protect  Product  from 
Contamination  Hazards.  Factors  associated  with  day-to-day  operations,  such  as 
housekeeping,  general  sanitation,  and  employee  hygiene  are  in  the  middle. 

Factors  dealing  with  attitudes  showed  the  least  amount  of  change.  These  are  good 
plants  to  begin  with,  and  it  is  difficult  to  detect  attitude  changes  in  the  short  time  period 
covered  by  the  study.  On  the  other  hand,  factors  dealing  more  directly  with  the 
production  of  a  safe,  quality  product  showed  the  greatest  number  of  changes.  These 
factors  dealt  primarily  with  management  initiated  programs  and  actions,  and  to  a  lesser 
degree,  employee  commitment  to  producing  a  wholesome  product. 
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TABLE  61:  QUALITATIVE  EVALUATION  OF  REFRIGERATED  FOODS  PLANT 
NUMBER:  RF  3 


PHASE  I  SCORE 

PHASE  HI  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

0 

0 

4 

3 

12 

3 

• 

13 

67 

201 

I 

29 

1 

30 

60 

1 

5 

5 

5 

SUM 

0 

3 

54 

42 

6 

105 

SCORE 

0 

12 

162 

84 

6 

264 

278 

BEFORE  AVG. 

2.65 

AFTER  AVG. 

2.S1 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  61  (CONTINUED) 


FACTOR 

ITEM 
NUMBER 

DESCRIPTION 

SCORE 
CHANGE 

POSITTVE  CHANGE 

I 

8 

Plant  design  and  layout 

3  to  2 

n 

9 

Plant  improvement  plan 

3  to  2 

rv 

1 

Employee  practices  to  prevent  cross-contaminaQon 

2  to  1 

IV 

3 

Plant  traffic  patterns 

3  to  2 

V 

la 

Plant  records  available 

3  to  2 

vm 

4b 

Short-term  improvements 

3  to  2 

vra 

4c 

Long-term  improvements 

3  to  2 

vm 

H 

Initiates  changes  in  product  safety  needs 

3  to  2 

vm 

4h 

Self-motivated  to  repair  facilities 

3  to  2 

vm 

4i 

Commits  adequate  resources  to  maintain  facilities 
and  produce  product 

3  to  2 

DC 

2c 

Layout  of  plant/product  flow 

3  to  2 

IX 

2j 

Prevent  product  contamination  with  foreign 
materials 

3  to  2 

XI 

2b 

Timely  update  of  programs 

3  to  2 

XI 

2c 

Laboratory  testing 

3  to  2 
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TABLE  62:  QUALITATIVE  EVALUATION  SUMMARY  OF  THE 
REFRIGERATED  FOODS  PLANTS 


PHASE  I  S.CORE 

PHASE  HI  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

1 

0 

0 

4 

6 

2 

8 

32 

3 

157; 

33 

190 

570 

2 

5 

74 

6 

85 

170 

] 

20 

20 

20 

SUM 

0 

6 

164 

107 

26 

303 

SCORE 

0. 

24 

492 

214 

26 

756 

792 

BEFORE  AVG. 

2.61 

AFTER  AVG. 

2.50 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...5  =  LOW 
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Cooked  Sausage  Plants: 

Cooked  Sausages  plant  1  (CS  1): 

CS  1  was  rated  on  101  of  the  possible  1 12  items.  The  average  score  of 
observations  during  Phase  I  was  2.66  and  2.44  during  Phase  III  (Table  63).  As  the 
averages  show,  CS  1  was  performing  on  average  above  medium  medium  and  continued  to 
improve  during  operations.  CS  1  had  positive  changes  for  23  of  the  101  items  observed. 
Positive  change  occurred  in  ten  of  the  11  factors,  with  five  changes  in  Plant  Management 
Measures  to  Protect  Product  from  Contamination  Hazards.  The  next  highest  number  of 
changes  (three)  came  in  General  Housekeeping.  CS  1  moved  from  low  medium  (just 
meeting  requirements)  to  medium  medium  (soundly  meeting  requirements)  on  10  items. 

Cooked  Sausages  plant  2  (CS  2): 

Out  of  the  112  total  items,  CS  2  was  scored  on  100  factors.  During  Phase  I  its 
average  score  was  3.08,  which  is  just  below  the  medium  medium  rating  of  soundly 
meeting  requirements  (Table  64).  The  average  score  of  observations  made  during  Phase 
III  improved  to  2.90.  CS  2  had  a  total  of  17  changes,  one  of  which  moved  from  medium 
medium  to  high.  This  change  was  for  Plant  Employee  Attitudes  about  Inspection.  Five 
items  moved  from  low  medium  to  medium  medium  and  1 1  moved  from  medium  medium 
to  high  medium.  Plant  Employee  Attitudes  about  Inspection  saw  movement  for  five  items 
while  Plant  Employee  Training  had  the  next  highest  number  of  factors  (four)  showing 
positive  movement. 

Cooked  Sausages  plant  3  (CS  3): 

CS  3  received  scores  for  101  of  the  112  items  (Table  65).  Its  average  score  during 
Phase  I  was  2.63,  which  is  above  the  medium  medium  score.  Observations  made  during 
Phase  III  saw  an  overall  improvement  to  2.54.  There  were  a  total  of  16  changes  in  the 
observations  from  Phase  I  to  Phase  III.  Of  the  16  changes,  12  were  in  a  positive 
direction,  while  four  were  in  a  negative  direction.  As  can  be  seen  in  the  table,  significant 
progress  was  made  in  some  areas,  while  significant  decline  was  made  in  other  areas.  Of 
the  four  negative  changes,  three  went  from  a  high  medium  to  a  low  medium  (just  meeting 
requirements).  On  the  other  hand,  four  moved  in  a  positive  direction  from  low  (not 
meeting  regulatory  requirements)  to  medium  medium  (soundly  meeting  regulatory 
requirements).  Three  of  those  four  jumped  two  increments. 

The  area  showing  greatest  improvement  was  Management  Programs  Providing  for 
Employee  Training.  Management  sent  one  employee  to  a  food  processing  workshop  and 
provided  summer  internship  to  a  college  student.  The  scoring  reflects  poorer  results  in  the 
factors  dealing  with  quality  control  and  TQC  implementation.  This  fact  is  partly 
explained  by  a  change  in  QC  personnel,  which  adversely  affected  the  plant's  QC 
programs.  Positive  changes  outnumbered  negative  changes  two  to  one. 
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TABLE  63:  QUALITATIVE  EVALUATION  OF  COOKED  SAUSAGES  PLANT 
NUMBER:  CS  1 


PHASE  I  S£ORE 

PHASE  m  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

0 

0 

4 

1 

10 

11 

44 

3 

42 

11 

53 

159 

2 

27 

2 

29 

58 

1 

* 

8 

8 

SUM 

0 

I 

52 

38 

10 

101 

0 

SCORE 

0 

4 

156 

76 

10 

246 

269 

BEFORE  AVG. 

2.66 

AFTER  AVG. 

2.44 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  63  (CONTINUED) 


FACTOR 

ITEM 
NUMBER 

DESCRIPTION 

SCORE 
CHANGE 

POSITIVE  CHANGE 

T 
1 

o 

z 

oiorage  or  panels,  supplies 

4  tO  3 

T 
1 

■5 
J 

v^oniroi  ux  prouuci  on  iioor 

1  tn  0 
J  10  L 

I 

5 

Sanitation  or  cleanup  staff 

3  to  2 

II 

6 

Product  contact  surfaces 

3  to  2 

II 

7 

Employee  welfare  facilities 

4  to  3 

III 

2 

Employee  clothing 

4  to  3 

IV 

1 

Employee  practices  to  prevent  cross- 
conidiHindiion 

3  to  2 

IV 

Plant  traffic  patterns 

4  10  J 

TV 
1  V 

A 
H 

iwaini.  sanitary  precautions 

A   tn  O 

4  tO  Z 

IV 

8 

Sanitary  storage  of  product  contact  items 

3  to  2 

VI 

2c 

TQM  involvement 

4  to  3 

VII 

lb 

Management  uses  TQM 

4  to  3 

VII 

lc 

Supervisors  use  TQM 

4  to  3 

VIII 

4i 

Commits  adequate  resources  to  maintain 
rac n  i ties  ana  proauce  proauci 

3  to  2 

rv 

LA 

1  K 
1  D 

T^^/~\ /H 1 1 ♦   tl aii/    inn/la  nlnnf 

rroauci  now  insiae  piani 

A  tn  ~i 

1 A 

1  A 

la 

Employee  traffic  inside  plant 

A  tn  1 

1  10  J 

TY 
1A 

Or- 

L,ayoui  oi  piani/proouci  now 

A  tn  1 

LA 

On 

rrocessmg  ecjuipmeni  uiaLmaineci 

"X  tn  0 
J  lO  Z 

IX 

2i 

Separation  of  raw  and  cooked 

3  to  2 

X 

2a 

Employee  commitment  to  safety 

3  to  2 

X 

2b 

Employees  take  independent  action 

3  to  2 

XI 

2d 

Lab  results  shared  with  FSIS 

2  to  1 

XI 

2e 

Lab  records  available  to  FSIS 

2  to  1 
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TABLE  64:  QUALITATIVE  EVALUATION  OF  COOKED  SAUSAGES  PLANT 
NUMBER:  CS  2 


PHASE  I SC.ORE 

PHASE  m  SCORE  » 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

0 

0 

4 

9 

5 

14 

56 

3 

m 

11 

1 

80 

240 

2 

6 

6 

12 

1 

0 

0 

SUM 

0 

9 

73 

17 

|  1 

100 

SCORE 

0 

36 

219 

34 

1 

290 

308 

BEFORE  AVG. 

3.08 

AFTER  AVG. 

2.90 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  64  (CONTINUED) 


FACTOR 

ITEM 
NUMBER 

DESCRIPTION 

SCORE 
CHANGE 

;*■■ 

POSITIVE  CHANGE 

I 

2 

Storage  of  pallets,  supplies 

4  to  3 

I 

5 

Sanitation  or  cleanup  start 

3  to  2 

n 

7 

Employee  welfare  facilities 

4  to  3 

n 

9 

Plant  improvement  plan 

3  to  2 

;  v 

lb 

Management  cooperate  -with  inspector  concerns  or 
requests 

3  to  2 

V 

lc 

Management  discusses  projects  with  inspectors 
during  planning 

3  to  2 

V 

Id 

Management  keeps  inspectors  informed 

3  to  2 

V 

le 

Management  demonstrates  commitment  to  safe, 
sanitary  food  production 

3  to  1 

V 

If 

Management  works  with  FSIS  management 

3  to  2  i 

VI 

la 

Sanitary  personal  hygiene 

4  to  3 

VI 

lb 

Sanitary  product  handling 

4  to  3 

VI 

2a 

Employee  refresher  training 

4  to  3 

VI 

2c 

TQM/employee  involvement 

3  to  2 

vn 

lb 

Management  use  of  TQM 

3  to  2 

vn 

lc 

Supervisors  use  of  TQM 

3  to  2 

vm 

1 

Management  response  to  problems 

3  to  2 

DC 

2h 

Management  ensures  use  of  GMPs 

3  to  2 
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TABLE  65:  QUALITATIVE  EVALUATION  OF  COOKED  SAUSAGES  PLANT 
NUMBER:  CS  3 


PHASE  I  SCORE 

PHASE  m SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

3 

1 

3 

7 

35 

4 

S 

3 

8 

32 

3 

33 

4 

1 

38 

114 

2 

3 

1 

33  I 

37 

74 

1 

11 

11 

11 

SUM 

3 

9 

40 

37 

12 

101 

SCORE 

15 

36 

120 

74 

12 

257 

266 

BEFORE  AVG. 

2.63 

AFTER  AVG. 

2.54 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  .  ..  5  =  LOW 
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TABLE  65  (CONTINUED) 


r  AC 1 OR 

P  1  V\K 

11  hM 
NUMBER 

DfcoCrur  I IUJM 

SCORE 
CHANGE 

NEGATIVE  CHANGE 

X 

lh 

TQM  implementation 

2  to  4 

XI 

2a 

Follows  TQC  procedures 

2  to  4 

X3 

3 

QA/QC  programs  or  systems 

2  to  3 

XI 

4 

QA/QC  department 

2  to  4 

XI 

5b 

Lab  staff  Training  level 

1  to  2 

POSITIVE  CHANGE 

I 

3 

Control  of  product  on  floor 

3  to  2 

n 

4 

Maintenance  of  cooler/ freezers 

3  to  2 

m 

1 

Employee  handwashing  practices 

4  to  3 

m 

3 

Employee  protective  covering 

4  to  3 

m 

4 

Company  hygiene  policies 

3  to  2 

7 

IV 

1 

Employee  practices  to  prevent  cross-contamination 

3  to  2 

IV 

8 

Sanitary  storage  of  product  contact  items 

4  to  3 

VI 

2d 

Outside  training  for  employees 

5  to  3 

VI 

2e 

Encourage  employees  to  become  certified  food 
handlers 

5  to  3 

VI 

2f 

Continuing  education  assistance 

5  to  4 

vn 

la 

Management  uses  employee  suggestions 

5  to  3 
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Cooked  Sausage  Plants  Summary: 

The  combined  scores  for  the  three  Cooked  Sausages  plants  are  presented  in  Table 
66.  As  displayed  in  the  table,  the  three  plants'  average  moved  from  2.79  during  Phase  I 
to  2.63  fdr  Phase  III.  Collectively,  these  plants  demonstrated  the  greatest  number  of 
changes  between  Phases  I  and  III.  Not  every  change  reflected  an  improvement,  as 
indicated  by  the  data  from  CS  3.  Some  of  the  changes,  both  positive  and  negative,  were 
significant  enough  to  be  two  scale  increments  away  from  the  original  observation.  There 
is  insufficient  information  to  deduce  whether  these  changes  are  a  consequence  of  HACCP 
implementation. 

In  the  two  plants  that  showed  only  positive  changes,  the  rates  of  change  were  23 
percent  and  17  percent,  respectively.  They  showed  improvement  in  most  factors; 
changes  were  primarily  from  one  medium  rating  to  the  next  medium  rating.  Neither 
plant  had  any  low  scores.  The  remaining  plant  presented  an  enigma,  in  that  it  made 
remarkable  progress  in  some  areas,  and  showed  significant  decline  in  others. 

When  combining  the  data  from  all  cooked  sausage  plants,  no  obvious  trends 
emerge.  In  fact,  some  of  the  factors  that  showed  the  greatest  change  in  the  refrigerated 
foods  plants  show  the  least  amount  of  change  in  the  cooked  sausage  plants.  The  opposite 
is  also  true;  of  the  total  302  observations  made  of  the  three  cooked  sausage  plants,  165 
(55  percent)  were  recorded  at  medium  medium  for  Phase  III.  Scores  of  medium  medium 
or  higher  were  recorded  for  280  (93  percent)  of  the  items  with  23  (8  percent)  rated  as 
high  as  opposed  to  22  items  (7  percent)  rated  at  low  medium  or  below.  Three  (less  than 
one  percent)  of  the  22  were  rated  as  low. 
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TABLE  66:  QUALITATIVE  EVALUATION  SUMMARY  OF  THREE  COOKED 
SAUSAGES  PLANTS 


DUACC  T  CPHDE 

rnAofc  1  bVtUKt 

PHASE  ffl  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

3 

1 

3 

7 

35 

4 

18 

33 

132 

3 

143  % 

26 

2 

171 

513 

2 

3 

1 

66 

2 

72 

144 

1 

19 

19 

19 

SUM 

3 

19 

165 

92 

23 

302 

SCORE 

15 

76 

495 

184 

23 

793 

843 

BEFORE  AVG. 

2.79 

AFTER  AVG. 

2.63 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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Poultry  Slaughter  Plants: 


Poultry  Slaughter  plant  1  (PS  1): 

Tht  scores  for  PS  1  on  the  99  items  observed  during  Phases  I  and  III  are 
presented  in  Table  67.  The  plant  showed  an  improvement  in  its  overall  average,  moving 
from  2.96  to  2.88.  Of  the  99  items  observed,  nine  positive  changes  and  one  negative 
change  were  observed.  Eight  items  moved  from  medium  medium  to  high  medium,  while 
the  remaining  positive  one  moved  from  low  medium  to  medium  medium.  The  negative 
change  was  from  high  medium  to  medium  medium  for  Employee  Attitude  about 
Inspection.  Plant  Management  Response  to  Problems  items  accounted  for  four  of  the 
nine  positive  changes. 

Poultry  Slaughter  plant  2  (PS  2): 

The  number  of  items  observed  for  PS  2  was  105  out  of  the  112  items  (Table  68). 
The  average  scores  for  Phases  I  and  III  were  2.48  and  2.45  respectively.  These  scores 
represent  an  average  better  than  medium  medium  (soundly  meets  requirements)  with 
movement  over  half  way  to  the  high  medium  rating  (exceeds  requirements).  Three 
positive  changes  (all  in  Plant  Employee  Personal  Hygiene)  were  observed,  with 
movement  from  high  medium  (exceeds  regulatory  requirements)  to  high  (greatly  exceeds 
regulatory  requirements).  For  those  items  that  showed  no  change,  PS  2  had  55  percent 
of  its  105  scores  at  the  medium  medium  rating,  36  percent  at  medium  high  and  7  percent 
at  high  score. 

Poultry  Slaughter  plant  3  (PS  3): 

PS  3  had  average  scores  for  the  100  items  observed  of  3.07  for  Phase  I  and  2.94 
for  Phase  III  (See  Table  69).  This  plant  showed  positive  movement  from  just  below  a 
medium  medium  rating  to  just  above  it.  A  total  of  10  positive  changes  were  observed 
with  three  items  moving  from  low  to  medium  medium  ratings.  Six  items  moved  from 
low  medium  to  medium  medium.  Items  in  Plant  Employee  Personal  Hygiene  had  the 
most  change  (4)  with  two  of  the  four  moving  from  low  to  medium  medium.  Of  the  100 
items  observed,  82  were  rated  at  medium  medium. 

Poultry  Slaughter  Plants  Summary: 

Poultry  slaughter  plants  showed  the  least  amount  of  change  between  Phases  I  and 
CI  (Table  70),  probably  because  poultry  slaughter  is  conducted  under  well  defined 
conditions,  especially  in  plants  operated  under  NELS  (New  Line  Speed  Inspection 
System)  or  SIS  (Streamlined  Inspection  System).  Poultry  slaughter  plants  operating  under 
NELS  must  have  an  approved  partial  quality  control  (PQC)  program  which  identifies 
critical  control  points,  and  is  operated  by  the  establishment's  quality  control  personnel. 
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TABLE  67:  QUALITATIVE  EVALUATION  OF  POULTRY  SLAUGHTER  PLANT 
NUMBER:  PS  1 


PHASE  I  SCORE 

PHASE  m  SCORE  * 

SUM 

SCORE 

N  . 

5 

4 

3 

2 

1 

5 

0 

0 

4 

i 

1 

4 

16 

3 

8 

87 

261 

2 

1 

O 

o 

ID 

1 

U 

n 

i  V 

SUM 

0 

3 

81 

15 

0 

99 

SCORE 

0 

12 

243 

30 

0 

285 

293 

BEFORE  AVG. 

2.96 

AFTER  AVG. 

2.88 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  67  (CONTINUED) 


FACTOR 

ITEM 

DESCRIPTION 

SCORE 

NEGATIVE  CHANGE 

IV 

6 

Sanitary  handling  of  offal 

2  to  3 

POSITIVE  CHANGE 

I 

7 

Microbiological  control  monitoring  program  for 
sanitation 

3  to  2 

V 

le 

Management  demonstrates  commitment  to 
safe/ sanitary  food  production 

3  to  2 

v 

if 

Svl UUECU1CUI   WUltU   WJLI1  i  J L— )  IUI    dUUIUVdJ  Ul 

programs,  info,  decisions 

J  to  *. 

v 

V 

2 

N^aintenance  of  inspection  office 

T  10  J 

vm 

2 

Planning  for  emergencies 

3  to  2 

vm 

4a 

Proactive,  preventive  plans 

3  to  2 

vm 

4f 

Management  initiates  changes  to  meet  plant  needs 

3  to  2 

vm 

4g 

Management  initiates  changes  to  meet  what 
inspection  sees  as  safety  needs 

3  to  2 

XI 

1 

Management  programs  exceed  minimal  regulatory 
requirements 

3  to  2 
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TABLE  68:  QUALITATIVE  EVALUATION  OF  POULTRY  SLAUGHTER  PLANT 
NUMBER:  PS  2 


PHASE  I  SCORE 

PHASE  ED  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

0 

0 

4 

0 

0 

3 

';  :■; 

57 

171 

2 

:  '■ «  Si; 
J8  ;| 

3 

HI 

] 

7 

/ 

7 

SUM 

0 

0^ 

57 

38 

10 

105 

SCORE 

0 

0 

171 

76 

10 

257 

260 

BEFORE  AVG. 

2.48 

AFTER  AVG. 

2.45 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 


TABLE  68  (CONTINUED) 


FACTOR 

ITEM 
NUMBER 

DESCRIPTION 

SCORE 
CHANGE 

POSITIVE  CHANGE 

m 

2 

Employee  clothing 

2  to  1 

m 

3 

Other  employee  protective  coverings 

2  to  1 

m 

4 

Company  hygiene  policies 

2  to  2 
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TABLE  69:  QUALITATIVE  EVALUATION  OF  POULTRY  SLAUGHTER  PLANT 
NUMBER:  PS  3 


PHASE  I  SCORE 

PHASE  ffl  SCORE  * 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

3 

3 

15 

4 

2 

6 

8 

32 

3 

1 

83 

249 

2 

5 

5 

10 

1 

1 

1 

1 

SUM 

0 

2 

91 

6 

1 

100 

SCORE 

0 

8 

273 

12 

1 

294 

307 

BEFORE  AVG. 

3.07 

AFTER  AVG. 

2.94 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  69  (CONTINUED) 


TTT\M 

I  i  EJVi 

NUMBER 

CHANGE 

POSITIVE  CHANGE 

I 

2 

Storage  of  pallets,  supplies 

4  to  3 

I 

'  5 

Sanitation  or  cleanup  staff 

3  to  2 

I 

7 

Micro  swabbing  for 'sanitation 

5  to  3 

n 

7 

Employee  welfare  facilities 

4  to  3 

m 

1 

Employee  handwashing  practices 

5  to  3 

m 

4 

Company  hygiene  policies 

5  to  3 

m 

5 

Visibility  of  policies  ; 

4  to  3 

m 

6 

Scope  of  hygiene  policies 

4  to  3 

IV 

8 

Sanitary  storage  of  product  contact  items 

4  to  3 

VI 

2c 

TQM/employee  involvement 

4  to  3 
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TABLE  70:  QUALITATIVE  EVALUATION  SUMMARY  OF  THE  POULTRY 
SLAUGHTER  PLANTS 


PHASE  I SCORE 

PHASE  ID  SCORE  • 

SUM 

SCORE 

N 

5 

4 

3 

2 

1 

5 

3 

3 

15 

4 

■  5 

7 

12 

48 

3 

21* 

9 

227 

681 

2 

1 

SO 

3 

54 

108 

I 

8 

8 

8 

SUM 

0 

5 

229 

59 

11 

304 

SCORE 

0 

20 

687 

118 

11 

836 

860 

BEFORE  AVG 

2.83 

AFTER  AVG. 

2.75 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...5  =  LOW 
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These  programs  resemble  HACCP  to  some  extent,  and  the  small  number  of  changes 
observed  in  the  poultry  slaughter  pilot  plant  is  not  surprising.  (Two  pilot  plants  are 
NELS). 

The  ratings  in  both  Phases  I  and  III  clustered  predominantly  in  the  medium 
medium  category.  No  pilot  plant  received  a  low  rating  in  any  subfactor  during  Phase  III. 
The  total  number  of  elements  evaluated  ranged  from  99  to  105,  with  the  number  of 
changes  being  either  three  or  ten.  The  bulk  of  the  changes  involved  employee  hygiene 
practices  and  general  housekeeping,  the  exception  being  a  downward  move  by  one  plant 
in  the  handling  of  offal.  The  overall  averages  for  the  three  plants  was  2.83  for  Phase  I 
and  2.75  for  Phase  III. 

The  plant  (PS  2)  with  the  fewest  number  of  changes  (three)  had  an  average  Phase 
III  score  of  2.45,  as  compared  with  2.88  and  2.94  for  PS  1  and  PS  3,  respectively.  PS  2 
also  operated  under  SIS,  rather  than  NELS,  but  does,  however,  use  PQC  programs  on 
the  processing  side  and  practices  an  informal  microbiological  monitoring  program. 

Nine  Plant  Summary: 

The  nine  plants  were  observed  a  total  of  909  times  for  the  11  factors  (Table  71). 
The  average  Phase  I  score  for  the  nine  plants  was  2.74  and  represents  an  above  medium 
medium  rating.  Phase  III  showed  a  positive  change  for  all  plants  in  their  overall 
averages.  The  nine-plant,  Phase  III  average  was  2.62.  There  were  125  changes,  ten  of 
which  were  in  a  negative  direction  (8  percent  [10/125]).  As  can  be  seen,  when  change 
occurred,  it  was  overwhelmingly  in  a  positive  direction.  Of  the  total  909  observations, 
nearly  13  percent  (115)  were  recorded  as  positive  changes.  The  medium  medium  rating 
was  observed  518  times  (57  percent),  while  change  to  the  medium  medium  rating  was 
observed  an  additional  40  times  (33  in  a  positive  direction  and  seven  in  a  negative 
direction).  As  seen  in  the  Phase  III,  60  of  the  909  items  (7  percent)  were  scored  as  high, 
which  represents  activities  that  greatly  exceed  requirements.  Only  3  items  were  rated  a 
low. 

Table  72  summarizes  the  rating  changes  for  items  of  the  11  factors  observed  at  the 
nine  plants.  Negative  changes  are  denoted  by  a  bracket  [■].  All  the  factors  showed  some 
improvement.  The  data  are  insufficient  to  draw  conclusions  concerning  the  impact  of 
HACCP  on  the  level  of  regulatory  compliance.    In  general,  plants  improved  in  their 
ratings.  Improvements  in  General  Housekeeping  were  widespread  and  were  much  easier 
to  identify  than  improvements  in  Plant  Management  Attitudes  Toward  Employee 
Supervision  because  intangible  issues  are  difficult  to  quantify.  In  general,  the  factors  that 
deal  more  directly  with  issues  encompassed  by  the  regulations  showed  greater  number  of 
changes  than  those  factors  that  dealt  with  intangible  issues. 
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TABLE  71:  QUALITATIVE  EVALUATION  SUMMARY  OF  REFRIGERATED 
FOODS,  COOKED  SAUSAGES,  AND  POULTRY  SLAUGHTER  PLANTS  (  3 
PLANTS  FOR  EACH  HACCP  AREA) 


PHASE  I  SCORE 

PHASE  m  SCORE  * 

SUM 

SCORE 

N 

4 

3 

2 

1 

5 

3 

1 

6 

10 

50 

4 

26 

27 

53 

212 

•2 

S18 

Do 

z 

588 

1764 

2 

3 

7 

m 

11 

211 

422 

1 

47 

47 

47 

SUM 

3 

30 

558 

258 

60 

909 

SCORE 

15 

120 

1674 

516 

60 

2385 

2495 

BEFORE  AVG. 

2.74 

AFTER  AVG. 

2.62 

*  1  =  HIGH;  2  =  HIGH  MEDIUM;  3  =  MEDIUM  MEDIUM;  ...  5  =  LOW 
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TABLE  72:  SUMMARY  OF  CHANGES  FOR  THE  11  FACTORS  OBSERVED  AT 
THE  NINE  HACCP  VOLUNTEER  PLANTS 


,                                                 Changes  in  factors  from  1st  to  2nd  visit 

RF  1 

RF  2 

RF  3 

CS  1 

CS  2 

CS  3 

PS  1 

PS  2 

PS  3 

I 

[1] 

3 

1 

3 

2 

1 

1 

3 

n 

1[2] 

2 

1 

2 

2 

1 

1 

m 

3 

2 

1 

3 

3 

4 

IV 

[1] 

2 

4 

2 

[1] 

1 

V 

1 

5 

3 

VI 

1 

2 

1 

4 

3 

1 

vn 

2 

1 

2 

2 

1 

vm 

1 

5 

1 

1 

4 

IX 

[1] 

1 

2 

5 

1 

X 

1 

5 

2 

[1] 

XI 

2 

2 

2 

[4] 

1 

[  ]  -  denotes  a  negative  change 
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HACCP  Pilot  Plant  Training: 


This  section  discusses  the  results  of  the  survey  administered  to  participants  who 
received  training  at  the  pilot  plants  (or  local  training  sites)  shortly  before  Phase  II 
HACCP  Implementation  activities  began.  As  stated  in  the  Methodology  section,  the 
primary  audience  for  the  training  was  FSIS  inspection  personnel.  Participants  included: 
from  FSIS,  inspectors,  relief  inspectors,  circuit  supervisors,  area  officials,  regional 
officials,  and  HACCP  data  collectors;  and  plant  managers,  production  supervisors,  and 
QC/QA  staff  from  the  plants. 

The  analysis  presents  the  aggregate  totals  for  all  101  participants  from  the  eight 
training  sessions.  No  assessment  was  made  for  the  training  at  PS  1  because  the  survey 
data  sheets  were  not  collected.  Table  73  shows  the  demographics  of  respondents  by 
process  category. 

TABLE  73:  CATEGORY  AND  NUMBER  OF  RESPONDENTS  FOR  THE  HACCP 
PILOT  TRAINING  SURVEY 


CATEGORY 

NUMBER 

TOTAL 

PERCENT 
OF  TOTAL 

REFRIGERATED 
FOODS 

COOKED  SAUSAGE 

POULTRY  SLAUGHTER 

1 

2 

3 

1 

2 

3 

1 

2 

3 

Meat  &  Poultry 
Industry 

0 

0 

4 

9 

1 

3 

0 

7 

6 

30 

30 

FSIS 

12 

5 

10 

7 

10 

7 

0 

10 

9 

70 

70 

Table  74  summarizes  the  responses  for  assessing  usefulness  of  the  training 
material  used.  The  plant-specific  plan  module  was  rated  as  being  useful/or  very  useful 
by  seven  out  of  every  eight  participants  (87  percent).  The  modules  for  monitoring  and 
evaluation  for  both  implementation  and  operational  phases  were  also  found  useful  by 
slightly  more  than  eight  out  of  every  ten  participants.  With  respect  to  understanding 
one's  role  (My  role),  nearly  seven  out  of  every  eight  respondents  thought  the  training  was 
useful  or  very  useful.  Overall,  a  significant  number  (more  than  seven  out  of  every  ten) 
rated  the  material  as  being  equal  to  or  better  than  useful.  In  only  one  area  (Inspector 
Assignment  Schedule)  did  more  than  20  percent  of  respondent  give  a  rating  below  the 
useful  level. 

Besides  rating  the  usefulness  of  the  training  materials,  participants  were  asked  to 
rate  the  effectiveness  of  the  training  materials.  Table  75  summarizes  their  responses  to 
the  14  items.  For  all  but  for  one  of  the  14  items  (Process  Deficiency  Record),  a 
rninimum  of  eight  out  of  every  ten  respondents  felt  the  materials  were  effective.  As  with 
the  usefulness  of  the  material,  the  vast  majority  of  respondents  rated  the  effectiveness  of 
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TABLE  74:  RESPONSES  AND  PERCENTAGES  FOR  TRAINING  MODULE'S 
USEFULNESS  RATING 


Usefulness  of  Training  Modules 

Module 

Not 

Useful/Effective 

Somewhat 
Useful/Effective 

Useful/Effective 

Very 
Useful/Effective 

Total  Response 

HACCP  Study 

2  (2%) 

11  (11%) 

47  (48%) 

38  (39%) 

98 

Plant-Specific  Plan 

9  (9%) 

45  (45%) 

47  (47%) 

101 

PBIS/HACCP 
relationship* 

1  (1%) 

13  (19%) 

27  (39%) 

28  (41%) 

69 

Supplemental 
Inspection  System 
Guide* 

1  (2%) 

9  (13%) 

24  (36%) 

33  (49%) 

67 

HACCP  Task  Plan** 

1  (3%) 

1  (3%) 

14  (44%) 

16  (50%) 

32 

Inspector  Assignment 
Schedule** 

8  (12%) 

11  (16%) 

29  (42%) 

21  (30%) 

69 

HACCP  Inspection 
Task  Schedule** 

2(7%) 

12  (39%) 

17  (55%) 

Process  Deficiency 
Record** 

1  (2%) 

14  (21%) 

27  (40%) 

26  (38%) 

68 

Sanitation  Report** 

3  (9%) 

15  (47%) 

14  (44%) 

32 

Monitoring  (during 
Phase  II). 

1  (1%) 

13  (14%) 

55  (57%) 

27  (28%) 

91 

Evaluation  (during 
Phase  II) 

1  (1%) 

13  (13%) 

56  (58%) 

27  (28%) 

97 

Monitoring  (during 
Phase  HI) 

1  (1%) 

10  (11%) 

52  (55%) 

32  (34%)  | 

95 

Evaluation  (during 
Phase  III) 

1  (1%) 

13  (14%) 

51  (54%) 

30  (32%) 

95 

My  role 

2  (2%) 

11  (12%) 

43  (46%) 

37  (40%) 

93 

Numbers  and  percents  are  for  Refrigerated  Foods  and  Cooked  Sausage  pilot  plant 
respondents. 

Numbers  and  percents  are  for  Poultry  Slaughter  pilot  plants. 
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TABLE  75:  RESPONSE  AND  PERCENTAGES  FOR  TRAINING  MATERIAL'S 
EFFECTIVENESS  RATING 


Effectiveness  Usefulness  of  Training  Modules 

Module 

Not 

Useful/Effective 

Somewhat 
Useful/Effective 

Useful/Effective 

Very 
Useful/Effective 

Total 
Responses 

HACCP  Study 

1  (1%) 

10  (10%) 

46  (47%) 

41  (42%) 

98 

Plant-Specific  Plan 

11  (11%) 

39  (39%) 

50  (50%) 

100 

relationship* 

1  (2%) 

10  (15%) 

29  (43%) 

28  (41%) 

68 

Supplemental 
Inspection  System 
Guide* 

1  (2%) 

9  (13%) 

23  (34%) 

34  (51%) 

(.7 

o/ 

HACCP  Task  Plan** 

1  (3%) 

16  (50%) 

15  (47%) 

32 

Inspector  Assignment 
Schedule* 

4  (6%) 

9  (13%) 

34  (50%) 

21  (31%) 

68 

HACCP  Inspection 
Task  Schedule** 

15  (48%) 

16  (52%) 

31 

Process  Deficiency 
Record* 

1  (2%) 

14  (21%) 

23  (34%) 

30  (44%) 

69 

Sanitation  Report** 

1  (3%) 

3  (9%) 

15  (47%) 

13  (41%) 

32 

Monitoring  (during 
Phase  II) 

1  /io/\ 

1  (1%) 

1  A   1 1  co/A 

Z7  \j\J7a) 

70 

Evaluation  (during 
Phase  II) 

16  (17%) 

54  (57%) 

25  (26%) 

95 

Monitoring  (during 
Phase  IE) 

13  (14%) 

51  (55%) 

29  (31%) 

93 

Evaluation  (during 
Phase  HI) 

14  (15%) 

51  (55%) 

28  (30%) 

93 

My  role 

12  (14%) 

44  (50%) 

32  (36%) 

Numbers  and  percents  are  for  Refrigerated  Foods  and  Cooked  Sausage  pilot  plant 
respondents. 

Numbers  and  percents  are  for  Poultry  Slaughter  pilot  plants. 
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the  materials  for  both  implementation  and  operational  phases  as  effective  or  higher  (i.e,  over 
80  percent  in  13  of  14  items). 

Fifty-seven  percent  of  the  respondents  rated  the  delivery  methods  used  (lecture,  hands- 
on  workshops^  etc.)  as  the  best  way  to  cover  the  subject  matter  for  each  module.  Forty 
percent  rated  the  delivery  methods  the  best  for  some  modules.  Suggestions  included 
providing  a  pre- informational  package,  more  hands-on  workshops,  videotapes,  telephone 
hotlines,  and  visits  from  Regional  Coordinators.  Nine  out  of  every  ten  respondents  rated  the 
trainers  as  effective  or  very  effective.  ("Trainers"  included:  Special  Team  presenters;  plant 
presenters;  and  others,  such  as  Processing  Inspector  Coordinators  (PIC's);  area  office  staff, 
and  Washington  staff).  Nearly  seven  out  of  every  eight  respondents  (86  percent)  felt  that  the 
subjects  presented  were  appropriate.  Additional  subjects  suggested  by  respondents  were  more 
basic  HACCP  information,  more  real  plant  problems  that  occur  during  operations,  review  of 
microbiological  test  results,  and  more  in-depth  training  on  monitoring.  Some  respondents 
suggested  deleting  or  reducing  the  repetition  of  previous  HACCP  history,  review  of  HACCP 
generic  plans,  time  spent  on  inspectors'  duties,  and  the  time  spent  on  each  topic. 

Headquarters  and  Regional  Coordinator  observers  of  the  training  sessions  agreed  that 
the  trainers  were  knowledgeable  about  the  subject  matter  and  presented  the  topics  in  a 
professional  manner.  They  gave  feedback  on  notebooks,  handouts,  style,  etc.,  to  the  trainers 
after  each  session.  For  example,  at  one  of  the  early  training  sessions,  it  was  suggested  that 
handouts  and  reference  material  should  not  be  given  to  participants  until  necessary,  to  prevent 
potential  confusion  due  to  reading  ahead  during  presentations.  At  subsequent  sessions,  the 
handouts  were  distributed  at  the  appropriate  times. 

The  observers  also  reported  that  the  subject  matter  (modules)  and  material  were 
appropriate  for  the  participants.  Regional  Coordinators  supported  these  observations  later 
during  monitoring  visits.  Inspection  personnel  were  able  to  perform  duties  and  understood 
their  responsibilities  in  the  pilot  plants.  Little  remedial  training  was  necessary  for  inspection 
employees.  Plant  personnel  demonstrated  an  adequate  working  knowledge  of  HACCP 
concepts  and  principles  at  most  of  the  pilot  plants.  It  should  be  noted  that,  at  three  of  the 
pilot  plants,  Agency  personnel  (non-inspection)  spent  significant  time  working  with 
responsible  plant  personnel  to  develop  acceptable  plant-specific  HACCP  plans. 

Overall,  the  training  was  believed  to  be  a  success  in  meeting  its  objectives  by 
providing  participants  with  an  understanding  of  the  seven  HACCP  principles,  the  generic 
versus  plant-specific  plans,  the  relationship  between  HACCP  and  the  PBIS  as  operated  in  the 
pilot  plants,  and  the  HACCP  study. 
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classification.  In  most  cases,  comparable  data  from  1992  and  1987  is  available  as  well.  The  data  files  on 
disc  one  and  two  are  provided  in  Microsoft  Access  database  format.  The  third  disc  contains  the  Adobe 
Acrobat  Portable  Document  Format  files  of  printed  reports  for  each  state  and  the  U.S.  Summary. 

Order  Number:  AC97CDV0L1 LPH 

Price:  $50.00  plus  handling  fee  for  U.S.,  Canada,  Mexico;  $100.00  all  others. 


The  Food  Stamp  Program 

The  Personal  Responsibility  and  Work  Opportunity  Reconciliation  Act  of  1996  dramatically  transformed 
and  continues  to  transform  the  food  assistance  landscape  in  the  United  States.  The  Act  cut  more  funds  from 
the  Food  Stamp  Program  than  it  did  from  any  other  program,  through  reductions  in  benefits  per  person  and 
restrictions  in  eligibility.  Despite  these  cuts,  food  stamps  now  have  a  more  prominent  role  in  the  post- 
welfare  reform  social  safety  net  because  the  largest  cash-assistance  entitlement  program,  Aid  to  Families 
with  Dependent  Children,  was  replaced  with  the  Temporary  Assistance  to  Need  Families  program,  a 
nonentitlement  program.  This  leaves  the  Food  Stamp  Program  as  one  of  the  only  remaining  entitlement 
program  available  to  almost  all  low-income  households. 

The  Changing  Food  Assistance  Landscape:  The  Food  Stamp  Program  in  a  Post-Welfare  Reform 
Environment. 

Order  Number:  ERSAER773LPH 

Price:  $14.00  plus  handling  fee  for  U.S.,  Canada,  Mexico;  $28.00  all  others. 


Reports  and  Studies  on  Kosovo  Conflict 
Available  Through  NTIS 

Learn  what  government,  academic,  political,  and  military  experts  and  analysts  have  diagnosed  as  the 
origins  of  the  Kosovo  conflict  and  what  measures  need  to  be  taken  for  peaceful  resolution: 
http://www.ntis.gov/kosovo.lttm 


Year  2000  Readiness  Disclosure 

NTIS  is  taking  steps  to  ensure  the  electronic  products  offered  to  our  customers  will  be  Y2K  compliant! 
Appropriate  modifications,  upgrades,  and  adjustments  are  being  made  to  current  NTIS  operating  systems  to 
ensure  Y2K  compliance.  For  more  information  on  our  progress,  access  Web  site 
http://www.  ntis.gov/y2k.htm. 


Prices  are  subject  to  change.  NTIS  Sales  Desk:  1-800-553-NTIS  (6847)  or  (703)  605-6000 


NTIS  World  Wide  Web  address:  http://www.ntis.gov 


Thank  you  for  your  order  from  NTIS!  Mf^FB^^^ 

The  products  listed  here  may  also  be  of  interest  to  you.  M^B  I  W^^Ps 
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Ideal  Prospects  for  Use  by  Associations, 
Universities,  and  Business  Services 

IRS  Tax  Practitioner  Mail  List 

The  IRS  Tax  Practitioner  Mail  List  is  an  active  file  of  approximately  380,000  tax  practitioners  who  have 
registered  with  the  Internal  Revenue  Service  to  receive  most  tax  forms,  publications,  and  instructions. 
These  professionals  are  involved  in  the  filing  of  federal  income  and  business  taxes,  and  the  file  is  updated 
continuously. 

Rent  this  list  to  announce  the  availability  of  tax-related  publications  or  to  offer  training  and  seminar 
opportunities.  The  list  also  provides  ideal  prospects  needing  forms,  office  supplies,  computer  software  and 
hardware. 

Ordering  Information:  Price  $45/M  for  4-up  Cheshire  labels;  S55/M  for  4-up  pressure-sensitive  labels. 
Minimum  order:  5,000.  Zip  ranges  -  add  $1/M  for  each  range.  Key  coding  -  add  $3/M.  Selections  available 
at  no  additional  charge  include  occupation,  state,  Nth  selection.  To  order  labels,  call  (703)  605-6488  or  fax 
(703)  605-6737. 

The  entire  file  is  available  on  magnetic  tape  for  $2,500.  To  order  this  tape,  call  the  NTIS  Sales  Desk  and 
quote  order  number  PB98-502032LPH. 

IRS  Enrolled  Agents  Mail  List 

This  is  an  active  file  of  enrolled  agents  who  are  eligible  to  practice  before  the  Internal  Revenue  Service. 
The  Enrolled  Agents  file,  compiled  by  the  U.S.  government,  contains  approximately  30,000  names  and  is 
updated  quarterly. 

Ordering  Information:  Price  $45/M  for  4-up  Cheshire  labels;  S55/M  for  4-up  pressure-sensitive  labels. 
Minimum  order:  5,000.  Zip  ranges  -  add  $1/M  for  each  range.  Key  coding  -  add  $3/M.  Selections  available 
at  no  additional  charge  include  occupation,  state,  Nth  selection.  To  order  labels,  call  (703)  605-6488  or  fax 
(703)  605-6737. 

The  entire  file  is  available  on  magnetic  tape  for  $625.  To  order  this  tape,  call  the  NTIS  Sales  Desk  and 
quote  order  number  PB93-500676. 


Preparing  for  a  New  School  Year  ... 

Today,  more  than  ever,  educators  and  those  in  training  capacities  are  faced  with  numerous  challenges.  One 
of  the  most  significant  challenges  is  how  to  present  day-to-day  curriculum  in  a  more  interesting  and 
informative  manner.  The  National  Audiovisual  Center  at  NTIS  is  here  to  help  you.  Exciting  multimedia 
training  aids  bring  an  element  of  new  interest  to  any  classroom.  And  our  selection  seems  endless  . . .  NAC 
offers  more  than  9,000  titles  in  a  range  of  subjects  as  varied  as  the  agencies  that  produce  them. 

The  Catalog  of  Educational  Multimedia  Products  pulls  together  listings  for  our  most  popular  categories 
such  as  Black  and  Women  Studies,  Counseling,  Drug  Prevention,  Health  and  Safety,  History  (from  the 
explorers  to  Post  World  War  II),  Languages,  Literature,  Music,  Science,  and  Social  Studies.  Also  included 
is  a  special  section  of  videos  in  Spanish. 

Be  ready  to  capture  your  students'  attention  this  year.  Take  a  minute  to  call  and  order  your  free  copy  of 
latest  Catalog  of  Educational  Multimedia  Products  (NTISPR1047LPH).  Or,  if  you  prefer,  download  your 
copy  from  NTIS  Web  site  http://www.ntis.gov/catalogs.htm. 

Prices  are  subject  to  change.  NTIS  Sales  Desk:  1-800-553-NTIS  C6847)  or  C703)  605-6000 


NTIS  World  Wide  Web  address:  http://www.ntis.gov 
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Each  week,  we  receive  more 
1  than  1,500  titles  from  both  the  U.S. 
government  and  international  sources  to 
add  to  our  collection  of  nearly  3  million 
information  products. 

Because  of  customer  request,  we  sift 
through  these  new  titles  and  categorize 
many  of  them  into  NTIS  Alert  subscription 
subject  areas. 


Twice  monthly,  we  can  send  you  the  titles 
and  summaries  of  these  reports.  All  you 
have  to  do  is  subscribe  to  the  NTIS  Alert 
in  your  subject  area  of  interest  and  sit 
back  and  wait  for  the  mail.  Then,  with 
the  latest  titles  and  summaries  in  hand,  you 
can  place  an  accurate  order  for  specific 
reports  you  need. 


Alert  Subject  Areas,  Annual 

Subscription  Prices,  and 
Subscription  Order  Numbers: 

Each  subscription  runs  12  months  beginning  with  the  date 
of  the  next  available  issue.  The  first  price  in  each  set  of 
parentheses  is  for  customers  in  the  United  States,  Canada, 
and  Mexico.  The  second  one  is  for  all  others.  Reference  the 
NTIS  order  number  beginning  with  "SUB"  when  calling  the 
NTIS  Subscriptions  Department. 


I 


Agriculture  &  Food 

($195/260)  [SUB-9098] 

Biomedical  Technology  & 
Human  Factors  Engineering 

($210/275)  [SUB-9095] 

Building  Industry  Technology 

($210/275)  [SUB-9089] 

Business  &  Economics 

($210/275)  [SUB-9096] 

Civil  Engineering 

($210/275)  [SUB-9050] 

Communication 

($210/275)  [SUB-9045] 


u 


Computers,  Control,  & 
Information  Theory 

($235/300)  [SUB-9062] 

Electrotechnology 

($210/275)  [SUB-9049] 

Energy 

($245/310)  [SUB-9097] 

Environmental  Pollution  &  Control 

($245/310)  [SUB-9068] 

Government  Inventions  for  Licensing 

($270/385)  [SUB-9090] 

Health  Care 

($210/275)  [SUB-9044] 

Manufacturing  Technology 

($265/330)  [SUB-9041] 

Materials  Sciences 

($220/285)  [SUB-9071] 

Ocean  Sciences  &  Technology 

($210/275)  [SUB-9047] 

Transportation 

($210/275)  [SUB-9085] 


The  NTIS  Subscriptions  Department  at  1-800-363-2068  or 
(703)  605-6060  can  answer  any  questions  and  save  you  time 
by  taking  your  order  over  the  phone.  Let  us  hear  from  you  soon! 
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Need  the  latest  U.S.  government 
study  results  in  a  specific  field? 

Want  to  save  costly  research  time, 
and  also  be  on  the  inside  track 
to  get  new  data  not  readily 
available  elsewhere? 

The  National  Technical 
Information  Service's  Alerts 
are  your  answer. 

We  have  the  information  you 
need,  and  we  make  it  easy  for 
YOU  to  get  it. 


"Alerts  are  available 
in  paper  copy. 
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NTIS  also  introduces  Electronic  Alerts, 
a  twice  monthly  E-mail  service 
which  offers  new  subject  areas. 

For  more  details,  visit  our  Web  site 
at  http://www.ntis.gov/alerts 


U.S.  DEPARTMENT  OF  COMMERCE 
Technology  Administration 
National  Technical  Information  Service 
Springfield,  VA  22161   (703)  605-6000 
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